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Computer-Based  Training  Development  and  Guidance  for  the  Army’s 
Unmanned  Aviation  Systems  Maintenance  Training  Division 

Introduction 

U.S.  Army  course  proponents  and  training  developers  often  face  tight  training  schedules 
that  limit  the  number  and  depth  of  topics  that  can  be  reasonably  covered  in  a  given  training 
program.  Excluding  one  topic  in  favor  of  including  or  expanding  on  another  may  lead  to 
deficiencies  in  essential  knowledge  and  skills  following  graduation  from  initial  military  training. 
Many  Army  courses  use  Computer-Based  Training  (CBT),  more  specifically  Interactive 
Multimedia  Instruction  (IMI)  to  enhance  the  efficacy  of  training  within  the  time  and  resources 
allocated  to  them.  IMI  is  a  specific  term  for  the  presentation  of  training  material,  via  computer, 
to  the  learner.  In  general,  Blankenbeckler,  Graves,  and  Wampler  (2014)  suggest  that  most  IMI 
efforts  appear  to  be  piecemeal  and  do  not  seem  to  follow  an  evidence-based  development 
strategy.  The  overarching  goal  of  this  current  research  was  to  develop  principles  and  guidance 
for  the  systematic  development  of  effective  IMI  that  can  be  broadly  applied  to  any  Army  course. 
This  research  supports  the  implementation  of  the  Army  Learning  Concept  (Department  of  the 
Army,  201 1)  by  providing  innovative  training  methods  to  build  and  deliver  highly  adaptable, 
versatile,  easy-to-access,  and  learner-centric  training  of  skills. 

The  intent  of  this  research  was  to  develop  CBT  guidance  and  tools  to  be  used  by  Army 
course  instructors  to  create  and  implement  effective  IMI,  expanding  upon  TRADOC’s  Army 
Educational  Processes  Pamphlet  350-70-7  (Department  of  the  Army,  2013).  Recent  ARI 
research  indicates  that  there  are  practical  CBT  principles  that  can  facilitate  the  design, 
development,  and  application  of  IMI  for  individualized  training  (Blankenbeckler,  Graves,  & 
Wampler,  2014).  Other  research  exists  for  applying  instructional  design  principles  to  CBT  (e.g., 
Bowden,  Gatskie,  &  Keller,  2001)  and  for  CBT  instructional  strategies  (e.g.,  Chen,  Barbee, 
Muraida,  &  Arnold,  1994).  ARI  researchers  reviewed  these  existing  principles  and  guidelines 
and  they  were  then  synthesized  and  extended  to  address  Soldier-centered  learning.  Products 
from  this  research  included  an  easily-referenced  learning  theory-based  CBT  Principles  Guide  for 
developing  effective  IMI  based  on  the  Army  Learning  Concept.  The  CBT  Principles  Guide 
focuses  on  generalizable  strategies  for  developing  and  using  IMI  and  is  supported  by  relevant 
research  literature  and  Subject  Matter  Expert  knowledge  and  experience.  Subsequently,  based 
on  early  reviews  of  the  CBT  Principles  Guide,  a  User’s  Guide  was  developed  to  offer  a  more 
streamlined  process  for  CBT  development. 


Method 

Based  on  the  CBT  design  principles  outlined  in  Graves,  Blankenbeckler,  Wampler,  and 
Roberts  (2016),  a  user-friendly  learning  theory-based  CBT  Principles  Guide  (Appendix  A)  was 
produced  to  assist  in-house  instructors  and  course  developers  with  increasing  the  learning 
effectiveness  of  their  IMI  materials  developed  in-house.  The  CBT  Principles  Guide  provided 
practical  examples,  based  on  learning  theory,  to  facilitate  the  integration  of  those  principles  into 
their  in-house  IMI  development  efforts.  Previous  research  identified  and  documented  some 
instructional  design  principles  to  enhance  the  learning  effectiveness  of  CBT  (Blankenbeckler, 
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Graves  &  Wampler,  2013,  2014;  Graves  et  al.,  2016).  Researchers  reviewed  the  existing  CBT 
principles  from  multiple  sources. 

Through  iterative  workgroup  discussions,  the  CBT  principles  were  organized  and 
grouped  based  on  different  factors  (e.g.,  anticipated  sequence  in  which  they  should  be  considered 
in  the  design  process,  significance  of  the  impact  of  adhering  to  or  not  using  the  principle,  and 
interrelationship  between  various  principles).  Since  there  was  no  intent  to  establish  a  definitive 
hierarchy  among  them,  the  CBT  principles  were  ultimately  assembled  to  facilitate  easy  access 
for  quick  reference.  A  very  brief  example  of  the  contents  and  organization  of  the  CBT  Principles 
Guide  is  provided  in  the  Results  section.  We  developed  and  applied  generalizable  principles  and 
guidelines  for  producing  effective  IMI  in  the  CBT  Principles  Guide.  Again,  the  CBT  Principles 
Guide  was  supplemented  with  a  User’s  Guide  that  provided  additional  material  that  expanded 
upon  the  design  guidance  of  the  CBT  Principles  Guide,  but  in  a  more  step-by-step  manner 
(Appendix  B). 


Description  of  Computer-Based  Training  Guidance 

One  effective  CBT  principle  is  to  define  the  learner’s  task.  Research  shows  that  a 
demonstration  of  the  particular  whole-task  similar  to  what  the  learners  should  be  able  to  do 
following  CBT  instruction  provides  a  basic  orientation  to  training  and  the  instructional  material, 
as  well  as  provides  motivation  for  the  learner  (Merrill,  2002).  Further  guidance  suggests  that  the 
CBT  should  show  the  learners  the  task  that  they  are  training  to  do  as  a  result  of  completing  the 
CBT.  It  is  imperative  that  CBT  engage  the  learner  at  the  real-world  problem  or  task  level 
(Schank,  2005).  One  technique  for  providing  whole-task  content  is  to  package  content  into 
smaller  pieces,  known  as  “chunking”,  that  makes  it  easier  for  the  student  to  assimilate  and  learn 
the  material  (Mayer,  2008). 

Another  important  principle  for  developing  Soldier-centered  CBT  may  be  the  learner’s 
control  of  content  that  varies  along  a  continuum  of  learner  control  (Garrison  &  Baynton,  1987). 
On  one  end  of  this  learner-control  continuum,  there  are  highly  structured  approaches  that  take  a 
learner  in  a  step-by-step  manner  through  the  learning  content  (i.e.,  designer-controlled 
instruction).  On  the  other  end  of  the  continuum,  learners  are  given  autonomy  in  selecting  the 
learning  content  for  themselves  (i.e.,  learner-controlled  instruction).  In  the  middle  ground  to  this 
continuum  are  techniques  that  support  well- structured  learning  designs  but  that  also  allow  for 
some  learner  control  and  autonomy  (Blankenbeckler  et  al.,  2014).  Further,  evidence  from  the 
military  and  other  research  settings  support  this  type  of  well-structured,  learner-controlled  design 
approach  depending  upon  the  knowledge,  skills,  and  experience  of  the  learners  (Dyer,  Singh,  & 
Clark,  2005). 

Garrison  and  Baynton  (1987)  describe  the  advantages  and  disadvantages  of  each  type  of 
controlled  design.  Designer-controlled  CBT  tends  to  apply  to  a  generalized  audience,  provide 
for  standardization  of  training,  is  easy  to  use  and  develop,  and  provides  a  high-level  of  guidance 
for  learners  that  are  new  to  a  domain.  Disadvantages  to  designer-controlled  CBT  are  that 
learners  progress  through  the  content  in  a  set  sequence,  take  more  time  to  complete  the  training, 
and  can  lose  motivation.  In  particular,  learners  with  higher  levels  of  knowledge,  skills  and 
experience  in  the  training  domain  may  be  negatively  affected  by  this  instructional  design. 
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Alternately,  some  advantages  of  learner-controlled  CBT  are  that  it  can  be  used  to  tailor  to 
individual  learners’  needs,  encourage  students  to  be  more  self-aware  during  the  learning  process, 
and  require  learners  to  be  more  autonomous.  Some  disadvantages  to  learner-controlled  CBT  are 
that  it  may  not  be  appropriate  for  some  topic  domains  or  for  learners  who  are  new  to  a  topic 
domain.  Without  appropriate  structure,  a  learner-controlled  design  may  cause  confusion  for 
some  learners,  have  a  negative  impact  on  learners  with  poor  learning  strategies  (i.e.,  those  who 
may  have  a  more  difficult  time  when  they  must  take  responsibility  for  their  learning),  and 
potentially  may  be  more  expensive  and  difficult  to  develop  because  of  the  specificity  required  to 
address  different  learners’  needs. 


Utilizing  the  Guides 

Previous  research  identified  and  documented  instructional  design  principles  to  enhance 
the  learning  effectiveness  of  CBT  (Blankenbeckler,  Graves  &  Wampler,  2013,  2014;  Graves  et 
al.,  2016).  The  systematic  consideration  and  application  of  sound  design  and  development 
principles  can  lead  to  more  effective  CBT.  Additionally,  putting  into  practice  these  CBT  design 
principles  better  supports  the  implementation  of  training  that  is  learner-centric,  more  effective, 
and  better  focused  on  identified  points-of-need.  However,  for  the  Army  to  realize  benefit  from 
them,  these  CBT  principles  need  to  be  adapted  to  address  the  military  audience  and  Army 
instructional  needs,  tailored  to  address  Soldier-centered  learning,  and  synthesized  with  practical 
constraints. 

The  CBT  Principles  Guide  developed  for  this  research  presents  the  principles  in  an  easy- 
to-comprehend  fashion.  The  CBT  Principles  Guide  was  originally  provided  in  Microsoft 
PowerPoint®  to  permit  embedding  of  examples  and  supporting  documents.  It  is  intended  as  a 
quick  reference  guide  for  in-house  CBT  developers  to  design  and  implement  effective  IMI. 

More  detail  is  provided  in  Appendix  A,  the  rest  of  this  section  describes  and  highlights  certain 
features  of  the  CBT  Principles  Guide. 

The  CBT  Principles  Guide  is  organized  into  three  main  sections.  In  addition  to  learning 
theory-based  principles,  other  useful  information  is  presented  to  assist  in-house  IMI  developers. 

•  The  ADDIE  (Analysis,  Design,  Development,  Implementation,  Evaluation)  Model  -  A 
framework  that  guides  the  design  and  development  of  training  (Branson  et  al.,  1975).  The 
ADDIE  model  is  an  accepted  framework  by  the  Department  of  the  Army  (2013).  Each  of 
the  phases  in  the  ADDIE  Model  is  briefly  explained  to  improve  understanding  of  how 
and  why  this  process  can  yield  effective  CBT, 

•  Design  and  Development  Principles  -  A  series  of  easily-referenced  fundamentals  that  can 
lead  to  more  effective  learner-centric  training.  Each  of  the  principles  are  introduced  and 
examples  are  provided  to  demonstrate  application  of  that  principle.  The  principles  focus 
on  generalizable  strategies  and  are  supported  by  relevant  research  literature,  expert 
knowledge,  and  practical  experience.  Incorporation  of  these  principles  in  CBT  should  be 
situational;  the  goal  is  to  design  effective  training,  and 
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•  Assessments  -  Learning  is  enhanced  when  it  is  assessed  regularly  and  meaningful 
feedback  is  provided  (Mayer,  2014).  Samples  of  useful  assessment  and  feedback 
techniques  are  provided. 

Design  principles  contained  in  the  CBT  Principles  Guide  come  from  previous  research 
(Blankenbeckler,  Graves  &  Wampler,  2013,  2014;  Graves  et  al.,  2016).  A  list  of  some  of  the 
principles  with  a  brief  explanation  of  each  is  provided  in  Table  1. 

Table  1. 

Examples  of  Design  Principles  for  CBT 

Principle _ Explanation _ 

Define  learning  task  What  the  learner  should  know  or  be  able  to  do  after  instruction 

Link  new  knowledge  Present  in  context  of  real-world  tasks  and  missions  (problem-based) 

Practice  Include  opportunities  to  practice  new  knowledge  and  skills;  present 

varied  situations  with  progressive  levels  of  difficulty 

Chunks  Provide  content  in  logical,  manageable  size  chunks 

Feedback  Provide  performance  feedback  and  tailored  remediation 

Promote/manage  essential  processing 

Segmenting  Present  materials  and  animation  in  learner-paced  segments 

Pre-training  Provide  essential  pre-training 

Modality  Consider  narrations  versus  printed  text 

Reduce  extraneous  processing 

Coherence  Avoid  presenting  extraneous  materials 

Signaling  Highlight  essential  materials 

Redundancy  Do  not  add  on-screen  text  to  narrated  animation 

Spatial  contiguity  Place  on-screen  text  next  to  corresponding  graphics 

Temporal  contiguity  Present  corresponding  narration  and  animation  at  the  same  time 
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Table  1. 

Examples  of  Design  Principles  for  CBT  ( Continued ) 


Principle 


Explanation 


Provide  a  meaningful  learning  experience  -  promote  generative  processing 

Multimedia  Present  words  and  pictures  rather  than  words  alone 

Personalization  Present  words  in  a  conversational  style  rather  than  formal  style 

Voice  Put  narrations  and  dialog  in  human  voice  rather  than  machine  voice 

Embodiment  Have  on-screen  agent  use  human-like  gestures  and  movements 

Image  Do  not  necessarily  put  the  static  image  of  agent  on  the  screen 


To  maximize  the  use  of  the  CBT  Principles  Guide,  it  should  be  viewed  in  the  Microsoft 
PowerPoint®  Slide  Show  viewing  mode  (see  Appendix  A  for  contact  information  on  how  to 
obtain  the  CBT  Principles  Guide  in  Microsoft  PowerPoint®).  Hyperlinks  are  provided  from  the 
Table  of  Contents  and  from  index  pages  of  design  and  development  principles  to  access  major 
sections  and  principles  of  the  CBT  Principles  Guide.  A  Return  button  is  also  provided  to  permit 
quick  return  to  the  page  of  origin.  Embedded  video,  documents,  and  audio  examples  can  be 
accessed  in  the  Microsoft  PowerPoint®  Slide  Show  viewing  mode  (see  example  on  page  8  of 
Appendix  A,  Slide  #14).  The  applications  may  open  over  the  Microsoft  PowerPoint®  Slide 
Show  or  appear  on  the  tray  of  the  viewing  window.  They  can  be  closed  without  closing  the 
Microsoft  PowerPoint®  Slide  Show  viewing  mode.  An  additional  assist  included  throughout  the 
CBT  Principles  Guide  displays  examples  of  what  ‘right’  and  ‘wrong’  might  look  like  in  an  IMI 
module.  A  sample  is  provided  on  page  14  of  Appendix  A,  Slide  #26.  As  the  CBT  Principles 
Guide  is  intended  to  be  an  easy-to-use  quick  reference  tool,  brief  explanations  and  examples  are 
provided  for  each  of  the  principles.  Also,  for  in-house  developers  who  want  to  learn  more  about 
the  various  principles,  a  list  of  references  is  included  with  the  CBT  Principles  Guide  for 
obtaining  further  information. 

To  assist  in-house  training  developers,  a  Learning  Effectiveness  Assessment  Worksheet 
(Appendix  C)  was  created  that  can  be  used  to  assess  their  use  of  the  design  principles  in  an  IMI 
module.  The  worksheet  is  focused  on  assessing  how  effectively  sound  design  principles  and 
techniques  have  been  integrated  to  their  IMI  to  enhance  learning.  Worksheet  users  should 
possess  an  understanding  and  working  knowledge  of  the  learning  principles  and  techniques.  The 
worksheet  is  designed  to  be  used  in  conjunction  with  the  CBT  Principles  Guide.  Previous 
research  has  shown  (Blankenbeckler,  Graves  &  Wampler,  2013,  2014;  Graves  et  al.,  2016)  that 
the  effectiveness  of  IMI  may  be  enhanced  by  practical  application  of  the  theory-based 
techniques,  guidelines,  and  methods  that  are  outlined  in  the  CBT  Principles  Guide  (Appendix  A) 
and  its  supplemental  User’s  Guide  (Appendix  B). 
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APPENDIX  A 

Computer-Based  Training  Principles  Guide 

The  CBT  Principles  Guide  is  designed  to  be  viewed  in  Microsoft 
PowerPoint®  Slide  Show  mode.  There  are  embedded  videos  and 
documents  that  can  be  accessed  from  that  mode.  The  following  pages  are 
merely  graphic  images  of  the  pages  in  the  CBT  Principles  Guide  and 

embedded  materials  are  not  accessible  here.  For  access  to  the 
hyperlinked  slidedeck,  please  contact  Dr.  Victor  Ingurgio: 
victor. j  .ingurgio.ci  v  @  mail.mil . 
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Training  (CBT): 

Principles  to  Increase  Learning  Effectiveness 
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General  I  nstructions  for  Use: 


This  Guide  is  provided  in  Microsoft  PowerPoint  to  permit  embedding  of 
examples  and  supporting  documents.  It  is  designed  for  quick  reference 
by  CBT  developers. 

Run  the  Guide  in  Slide  Show.  Hyper  navigation  is  provided  to  and  from 
the  Table  of  Contents  and  selected  Index  slides.  Video,  MS  Word 
documents,  and  audio  examples  can  be  accessed  in  Slide  Show  viewing 
mode  with  the  click  of  the  mouse.  The  applications  may  open  over  the 
PowerPoint  Slide  Show  or  appear  on  the  tray  of  your  viewing  window. 
They  can  be  closed  without  shutting  down  Slide  Show  mode. 

Hyperlinks  are  provided  from  the  Table  of  Contents  and  I  ndex  of  Design 
and  Development  Principles  slides  to  access  major  Sections  and 
Principles  of  the  Guide.  Return  button  navigation  is  also  provided  to 
permit  quick  return  to  the  page  of  origin. 


References  are  cited  throughout  this  Guide  using  superscript  end  note 
numbers  (#).  These  references  and  source  documents  can  be  found  on  Slide 
53  listed  in  numerical  order. 
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I  ntroduction 


U.S.  Army  course  proponents  and  training  developers  often  face 
tight  training  schedules  that  limit  the  number  and  depth  of  topics 
that  can  be  reasonably  covered  in  a  given  training  program. 
Expanding  the  course  is  seldom  an  option.  Excluding  one  topic  in 
favor  of  including  or  expanding  another  may  lead  to  deficiencies 
in  essential  knowledge  and  skills  following  graduation  from  initial 
military  training,  leader,  or  functional  courses.  Many  Army 
courses  have  turned  to  computer-based  training  (CBT)  to 
complement  or  supplement  training  within  the  time  and  other 
resources  allocated  to  them.  Some  schools  and  courses  have 
implemented  a  program  of  in-house  development  to  better  meet 
their  needs. 

While  an  excellent  initiative  and  fruitful,  these  CBT  efforts  are 
sometimes  less  effective  than  desired.  Course  instructors  or 
developers  create  the  CBT  on  their  own  without  the  benefit  of 
clear  instructional  guidance  or  a  thorough  consideration  or 
application  of  proven  design  principles.  Constrained  by  time, 
available  talent,  and  other  resources,  they  do  not  necessarily 
follow  a  comprehensive  development  strategy  nor  do  their 
products  undergo  a  rigorous  design  scrutiny  prior  to 
implementation.  While  these  courses  are  beneficial,  the 
systematic  consideration  and  application  of  sound  design  and 
development  principles  can  lead  to  more  effective  CBT. 
Additionally,  putting  into  practice  these  design  principles  better 
supports  the  implementation  of  the  Army  Learning  Model, 
providing  training  that  is  learner-centric,  more  effective,  and 
better  focused  on  identified  points- of- need. 

Training  research  has  identified  practical  principles  that  can 
improve  the  effectiveness  of  CBT  for  individual  training. 

However,  these  principles  and  guidelines  need  to  be  adapted  to 
address  the  military  audience  and  Army  instructional  needs, 
tailored  to  address  Soldier-  centered  learning,  and  synthesized 
with  practical  constraints.  This  Guide  bridges  that  gap, 
presenting  the  principles  through  examples,  guidance,  and  tools 
that  may  be  used  by  Army  trainers  and  training  developers  to 
design  and  implement  effective  CBT. 
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The  Guide  is  organized  into  three  main  sections. 


•  ADDI E  Model  -  A  framework  that  provides  a  guideline  for  the 
design  and  development  of  training.  Each  of  the  phases  in  the 
ADDIE  Model  (analysis,  design,  development,  implementation, 
and  evaluation)  are  briefly  explained  to  improve  understanding  of 
how  and  why  this  process  can  yield  effective  CBT. 

•  Design  and  Development  Principles  -  A  series  of  easily- 
referenced  fundamentals  that  can  lead  to  more  effective  learner¬ 
centric  training.  Each  of  these  principles  is  introduced  and 
samples  or  examples  are  provided  to  demonstrate  application  of 
the  principle. 

•  Assessments  -  Learning  is  enhanced  when  learning  is  assessed 
regularly  and  meaningful  feedback  is  provided.  Samples  of 
useful  assessment  and  feedback  techniques  are  provided. 

This  Guide  and  its  principles  are  intended  to  assist  in  the  design  and 
development  of  effective  CBT.  The  principles  focus  on  generalizable 
strategies  and  are  supported  by  relevant  research  literature,  expert 
knowledge,  and  practical  experience.  Note  that  incorporation  of  the 
Principles  should  be  situational;  the  goal  is  to  design  effective 
training. 
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Principles  to  Guide  the  Design  and 
Development  of  CBT 


Table  of  Contents 

ADDIE  process  (Slides  6- 11) 

Design  and  Development  Principles  (Index  of  Design  and 

Development  Principles  -  Slides  12-13) 

□  Define  the  learning  task  -  what  the  learner  should  know  or  be 
able  to  do  after  instruction 

□  Link  new  knowledge  and  skills  to  real  world  tasks  and 
missions  in  context  (problem  based) 

□  Encourage  practice  of  new  knowledge  and  skills 

□  Provide  content  in  logical  chunks 

□  Provide  a  meaningful  learning  experience- promote  generative 
processing 

Assess  student  learning  (Slides  46-50) 
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The  ADDIE  Model 


The  ADDI E  model  is  an  accepted  framework  that  incorporates  processes 
used  by  instructional  designers  and  training  developers.1  It  provides  a 
guideline  for  the  design  and  development  of  computer  based  training 
and  performance  support  tools.  It  derives  its  name  from  its  five  phases: 

•  Analysis 

•  Design 

•  Development 

•  Implementation 

•  Evaluation 

Over  time  this  model  has  been  revised  and  is  now  more  dynamic  and 
interactive  than  the  original  design.  Revision,  adjustment  and  fine  tuning 
occur  throughout  the  process,  and  Evaluation  have  become  central  to 
each  phase.  The  illustration  depicts  a  simplified,  revised  version  of  the 
ADDIE  Model. 
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Analysis  Phase  of  ADDIE 


The  Analysis  Phase  clarifies  and  identifies  the  instructional  target,  the 
environment,  problems,  and  objectives.  During  this  phase,  you  should 
answer: 

•  Who  are  the  learners  and  what  are  their  characteristics?  (Who  are 
your  students  and  what  entry  level  knowledge  and  skills  will  they 
enter  the  course  with?) 

•  What  is  the  desired  new  behavior?  (What  knowledge,  skills,  level 
of  proficiency,  and  performance  are  desired  upon  completion  of 
the  lesson  or  course?) 

•  What  types  of  learning  constraints  exist? 

•  What  are  the  delivery  options?  (Match  the  instructional 
methodologies  and  media  to  the  tasks  to  be  trained.) 

•  What  are  the  pedagogical  considerations? 

•  What  Army  Learning  Model  and/or  adult  learning  theory 
considerations  apply?  Additionally,  you  must  determine  what  you 
can  do  within  time,  talent,  budget,  and  other  resource  constraints. 


TRADOC  Regulation  350-70-7 
(2013)  no  longer  prescribes 
particular  instructional 
methodologies  for  a  given  subject  or 
task.  That  determination  depends  on 
the  learning  outcomes,  target 
audience,  learning  strategy, 
resources,  and  design.  However,  to 
assist  with  the  analysis  process,  a 
previously  used  Matrix  of 
I  nstructional  Strategies  and 
Methods  for  Delivering 
Instruction  is  provided. 


Matrix  of  Instructional 
Strategies  and 
Methods  for 
Delivering  Instruction 

m 

Matrix 

itructional  Strategi 

(Open  the  document  with  a 
mouse  click.) 


The  remainder  of  this  Guide  is  focused  on  the  premise  that  your 
decision  was  to  select  the  I  nstructional  Method  of 
I  ndividualized,  Self-Paced  I  nstruction  or  some  other  form  of 
instruction  using  CBT. 
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Design  Phase  of  ADDIE 


The  Design  Phase  deals  with  the  creation  of  learning 
objectives,  assessment  instruments,  exercises,  content, 
subject  matter  analysis,  lesson  planning,  and  media 
selection.  The  design  phase  should  be  specific  and 
systematic.  Follow  a  logical,  orderly  method  of  identifying, 
developing  and  evaluating  a  set  of  planned  strategies 
targeted  for  attaining  the  instructional  goals.  Elements  of 
the  instructional  design  plan  must  be  executed  with 
attention  to  details. 


•  Document  the  instructional,  visual,  and  technical 
design  strategy. 

•  Apply  instructional  strategies  according  to 
intended/desired  behavioral  outcomes  by  domain 
(cognitive,  affective,  and  psychomotor). 

•  Design  the  graphic  user  interface  that  will  provide  the 
navigation,  facilitate  user  interaction  with  the  media, 
and  set  the  context  for  the  user's  learning  experience. 

•  Create  a  prototype  (or  prototypes)  (Make  sure  your 
ideas  work  before  you  step  into  full  production.) 

•  Apply  the  visual  design  and  motif.  (The  combining  of 
words,  symbols,  page  layout,  and  images  provide  this 
graphic  design.  It  creates  a  visual  representation  of 
the  ideas  and  messages  conveyed  by  the  media.  This 
visual  design  becomes  the  "stage"  and  setting  that 
provide  much  of  the  context  and  orientation  for  the 
user's  learning  experience.) 


This  Guide  provides  principles  and  examples  useful  in  the 
Design  and  Development  Phases  of  CBT. 
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Development  Phase  of  ADDIE 


In  the  Development  Phase,  instructional  designers  and 
developers  create  and  assemble  content  assets 
blueprinted  in  the  Design  Phase. 

In  this  phase,  the  designers  create  storyboards,  graphics, 
and  audio. 

Programmers  develop  and  integrate  graphics,  audio, 
video  and  other  content. 

Testers  check  materials  for  fidelity  -  navigation,  correct 
feedback,  and  other  issues  -  and  identify  problems  for 
edits  and  debugging. 

The  project  is  reviewed  and  revised  according  to 
feedback. 


This  Guide  provides  principles  and  examples  useful  in 
the  Design  and  Development  Phases  of  CBT. 
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Implementation  Phase  of  ADDIE 


The  Implementation  Phase  prepares  training  facilitators  and 
learners  to  use  the  materials. 

For  example,  procedures  for  training  facilitators  and  learners  are 
put  in  place.  Training  for  facilitators  may  cover  the  course 
curriculum,  learning  outcomes,  method  of  delivery,  and  testing 
procedures.  Preparations  for  learners  may  include  training  them  on 
new  tools  (software  or  hardware)  and  enrolling  or  registering 
students. 

Implementation  includes  evaluation  of  the  design.  It  may  include  a 
pilot  or  trial  course  to  provide  a  final  troubleshooting  of  the 
courseware  and  procedures. 

Course  leaders  and  managers  may  also  ensure  that  training 
materials,  books  and  manuals,  equipment  and  tools,  CD-ROMs, 
and  software  are  in  place. 

Additionally,  they  may  make  certain  that  the  learning  applications, 
website  functions,  and  learning  management  systems  are  in  place 
and  ready,  if  required. 
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Evaluation  Phase  of  ADDIE 


The  Evaluation  Phase  has  two  aspects:  formative  and  summative. 

Formative  assessments  occur  in  each  phase  of  the  ADDI E  process. 
Formative  assessments  may  include  diagnostic  testing  or  a  wide 
range  of  formal  and  informal  assessment  procedures  employed  by 
trainers  during  the  learning  process  in  order  to  modify  teaching  and 
learning  activities.  They  have  a  goal  of  enhancing  the  course  or 
courseware  to  increase  student  learning,  proficiency,  or  attainment. 
They  may  involve  qualitative  feedback  (rather  than  scores)  focused 
on  the  details  of  instructional  content,  instructional  methods,  and 
course  or  lesson  performance. 

Summative  assessments  are  focused  on  the  learner  and  summarize 
their  development  or  learning  at  a  particular  time  or  phase  of  the 
course.  They  are  an  assessment  of  learning  in  contrast  to  the 
formative  assessment,  which  is  assessment  for  learning. 

Summative  assessments  provide  information  on  the  lesson  or 
course's  efficacy  (its  ability  to  do  what  it  was  designed  to  do). 
Summative  evaluation  judges  the  worth,  or  value,  of  a  lesson  or 
course  at  its  conclusion. 

Evaluation  continues  after  instructional  implementation,  providing 
feedback  for  minor  adjustments  and  "tweaks"  as  well  as  formal 
reviews  to  assure  that  the  course  content  remains  current  and 
continues  to  be  effective  over  time. 
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Evaluation  Phase  of  ADDIE 


The  Evaluation  Phase  has  two  aspects:  formative  and  summative. 

Formative  assessments  occur  in  each  phase  of  the  ADDI E  process. 
Formative  assessments  may  include  diagnostic  testing  or  a  wide 
range  of  formal  and  informal  assessment  procedures  employed  by 
trainers  during  the  learning  process  in  order  to  modify  teaching  and 
learning  activities.  They  have  a  goal  of  enhancing  the  course  or 
courseware  to  increase  student  learning,  proficiency,  or  attainment. 
They  may  involve  qualitative  feedback  (rather  than  scores)  focused 
on  the  details  of  instructional  content,  instructional  methods,  and 
course  or  lesson  performance. 

Summative  assessments  are  focused  on  the  learner  and  summarize 
their  development  or  learning  at  a  particular  time  or  phase  of  the 
course.  They  are  an  assessment  of  learning  in  contrast  to  the 
formative  assessment,  which  is  assessment  for  learning. 

Summative  assessments  provide  information  on  the  lesson  or 
course's  efficacy  (its  ability  to  do  what  it  was  designed  to  do). 
Summative  evaluation  judges  the  worth,  or  value,  of  a  lesson  or 
course  at  its  conclusion. 

Evaluation  continues  after  instructional  implementation,  providing 
feedback  for  minor  adjustments  and  "tweaks”  as  well  as  formal 
reviews  to  assure  that  the  course  content  remains  current  and 
continues  to  be  effective  over  time. 
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Index  of  Design  and  Development  Principles  for  CBT 


[Instructions:  Major  principles  are  hyperlinked  for  ease  of  access.] 

•  Define  the  learning  task  and  objective  -  what  the  learner  should 
know  or  be  able  to  do  after  instruction.  (Slide  14) 

•  Link  new  knowledge  and  skills  to  real  world  tasks  and  missions  in  context 
(problem  based)  (Slide  15) 

•  Encourage  practice  of  new  knowledge  and  skills  (Slides  16  -  18) 

□  Provide  varied  situations,  progressive  levels  of  difficulty,  and  lessen 
multi-facetted  instructional  support  and  coaching  (scaffolding)  as 
familiarity  with  the  task  and  steps  increase. 

□  Provide  performance  feedback  and  tailored  remediation. 

•  Provide  content  in  logical  chunks  (Slides  19  -  31) 

□  Promote/manage  essential  processing 

>  Segmenting  -  Present  materials  and  animation  in  Learner-paced 
segments 

>  Pre-training  -  Provide  essential  pre-training 

>  Modality  -  Present  narrations  vs.  printed  text 

□  Reduce  extraneous  processing 

>  Coherence  -  Avoid  presenting  extraneous  materials 

>  Signaling  -  Highlight  essential  materials 

>  Redundancy  -  Do  not  add  on-screen  text  to  narrated  animation 

>  Spatial  Contiguity  -  Place  on-screen  text  next  to  corresponding 
graphics 

>  Temporal  Contiguity  -  Present  corresponding  narration  and 
animation  at  the  same  time 
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Index  of  Design  and  Development  Principles  for  CBT 

(continued) 

Provide  a  meaningful  learning  experience  -  promote  generative  processing  (Slides 
32  -  45) 

□  Multimedia  -  Present  words  and  pictures  rather  than  words  alone 

□  Personalization  -  Present  words  in  a  conversational  style  rather  than 
formal  style 

□  Voice  -  Put  narrations  and  dialog  in  human  voice  rather  than  machine  voice 

□  Embodiment  -  Have  on-screen  agent  use  human- like  gestures  and 
movements 

□  I  mage  -  Do  not  necessarily  put  the  static  image  of  agent  on  the  screen 
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Define  the  Learning  Task,  Objective,  and  Expected  Student  Performance 


Et  Is  common  practice  to  state  learning 
objectives  at  the  beginning  of  lesson 
materials,  for  example,  learning 
objectives  usually  take  a  form  like: 
"The  learner  will  be  able  to  ... 
However  Just  Stating  the  learning  task, 
objective,  and/or  performance 
standard  IS  often  not  adequate. 


This  video  dip  provides  a  model,  an  example  of 
a  lesson  for  the  tactical  task  -  Conduct 
immediate  suppression.  This  example 
provides  a  motivating  introduction  that 
demonstrates  the  whole  task.  3t  then  presents 
the  learning  objective. 


Learning  objectives  presented  as  an 
abstract  statement  are  often  only 
understood  by  the  learner  following  the 
instruction.  They  do  little  to  set  the 
context  or  engage  the  learner's 
attention.  Research  shows  that  a 
demonstration  of  the  particular  whole 
task  similar  to  what  the  learners  will 
be  able  to  do  following  instruction 
provides  a  much  better  orientation  to 
training  and  Instructional  material  as 
well  as  provides  motivation, * 

Show  the  [Earner  the  task  that  they  willl 
be  a  bit  to  do  or  the  problem  they  will 
be  able  to  solve  as  a  result  of 
comptebng  a  module  or  course. 

Engage  them  at  the  real  world  problem 
or  task  level,  not  just  In  a  task  step, 
operation,  action,  or  procedure.1 
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Link  New  Knowledge  to  Real  World  Tasks  and  Missions 


Training  research  shows  that  learners 
assume  ownership,  that  is  apply 
themselves  to  learning  and  show 
enthusiasm,  when  problems  to  be 
solved  are  interesting,  relevant,  and 
engaging.  I  ntroduce  the  whole  task, 
problem,  and/or  real-world  application 
-  EARLY.2 

WHAT  NOT  TO  DO 
Topic  Centered  Instruction  often 
introduces  subtasks  or  components  of 
a  task  in  isolation.  The  subtask  or 
component  is  never  related  to  the 
whole  task.  Trainers  sometimes  tell 
the  students,  "You  won't  understand 
this  now  but  later  it  will  be  really 
important  to  you."  Students  are 
taught  the  prerequisites  before  any 
introduction  of  or  link  to  the  real  world 
application/whole  task. 

WHAT  TO  DO 

Problem-Centered  Instruction  stands 
in  contrast  to  this  watered-down 
approach.  Learning  is  best  promoted 
when  learners  are  engaged  in  solving 
a  real-world  problem.  The  problem 
includes  a  wide  range  of  activities,  the 
most  critical  characteristic  being  that 
the  activity  is  part  of  some  whole  task 
rather  than  only  a  component  of  a 
task  and  that  the  task  is  representa¬ 
tive  of  situations  that  the  learner  will 
encounter  in  duties  and  assignments 
following  instruction. 


The  early  linking  of  new  knowledge  and  skills 
to  real  world  tasks  and  missions  provides 
motivation,  an  immediate  reason  to  learn  new 
skills  or  gain  new  knowledge,  as  well  as  the 
context  for  future  use  and  application. 

Learning  to  complete  a  whole  task  involves 
four  levels  of  instruction  and  student 
performance: 

1.  The  problem 

2.  The  tasks  required  to  solve  the  problem 

3.  The  operations  that  comprise  the  tasks 

4.  The  actions  that  comprise  the  operations 

The  most  effective  instruction  engages 
students  in  all  four  levels  of  performance. 

It  provides  a  context  and  explains  to  the 
student  early  how  the  tasks,  operations, 
and  actions  interrelate  and  why  they  are 
important. 


This  vintage  erector  set  employed  this 

Principle. 

It  taugnt  the  "future  engineer": 

•  The  use  and  value 
of  basic  building 
components 

•  How  components 
contributed  to 


geometric  shapes 

•  The  engineering 
characteristics  of 
basic  geometric 
shapes 

•  The  relationship  of 
these  shapes  to 
the  successful 


construction  of 
fundamental 


construction  of 
large  complex 
structures,  such 
as  bridges  and 
buildings. 
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Encourage  Practice  of  New  Knowledge  and  Skills 


Learning  is  doing.2  The  old  saying  that 
"practice  makes  perfect"  has  validity. 
However,  for  practice  to  be  effective  it 
should  follow  some  basic  rules: 

•  Students  learn  when  they  use  their 
new  knowledge  and  skills  to  solve 
problems. 

•  Practice  (and  tests)  should  be 
consistent  with  stated  learning 
objectives  and  performance 
standards  of  the  task,  for  example: 

□  Learners  recall  or  recognize 
information  about  or  related  to 
the  problem  (or  subtask  or 
component  of  the  problem). 

□  Learners  locate,  name,  describe, 
or  identify  components  or  parts. 

□  Learners  identify  new  examples 
of  components,  parts,  or 
elements. 

□  Learners  perform  procedures 

□  Learners: 

>  Predict  the  consequences  of 
an  action,  when  given  the 
conditions  and/or 
circumstances. 

•  Troubleshoot  or  determine 
a  fault  when  given  an 
unexpected  consequence  or 
failure. 


•  Practice  and  problem  solving  should 
be  initially  supported  by  scaffolding  - 
error  detection,  feedback,  and  coach¬ 
ing.  However,  coaching  and  other 
instructional  support  should  be 
gradually  withdrawn  as  the  student 
gains  proficiency  and  confidence. 

Note  that  all  of  the  examples  have  the 
learner  doing  something,  not  memorizing 
and  regurgitating  what  they  were  told. 

Making  mistakes  in  practice  is  okay. 
Students  learn  from  making  errors,  if 
they  are  shown  how  to: 

•  Recognize  the  error 

•  Recover  from  the  error 

•  Avoid  the  error 

Applying  new  knowledge  in  a  single 
problem  is  insufficient,  especially  when 
learning  a  cognitive  skill.3  Practice 
should  provide  a  range  of  application 
opportunities  with  varied  conditions, 
situations,  and  increasing  levels  of 
difficulty  for  the  new  learner. 
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Encourage  Practice  of  New  Knowledge 
and  Skills  (continued) 


Allow  the  student  to  practice  the  skill,  not  the  same  problem  or  context. 


What  practice  isn't. 

Practice  is  not  introducing  a  concept  or 
step  using  a  demonstration  in  a  context, 
then: 

Being  provided  examples  In  the  same 
or  very  similar  context. 

Performing  the  same  step  (repetition) 
again  and  again  In  the  same  context. 

Working  through  a  quiz  or  check  on 
learning  in  the  same  context  or 
situation. 

Encountering  the  same  or  similar 
problem  and  context  in  an  end  of 
section  or  final  exam. 

The  learner  now  may  understand 
how  to  solve  one  (1)  problem 
with  the  new  skill. 


What  practice  should  be* 

Practice  is  having  a  concept  or  step 
introduced  using  a  demonstration  in  a 
particular  context. 

Providing  examples  in  similar  but  varied 
contexts. 

Performing  the  steps  to  solve  different 
related  problems  of  increasing  difficulty,  in 
varied  contexts  ,| 

Working  through  different  but  related 
problems  in  a  quiz  or  check  on  learning. 

Encountering  different  but  related 
situations  and  problems  in  an  end  of 
section  or  final  exam. 


n 

Intro 


Pi  □  f . 

' - 1  UplTflin 


Reps 


Retrain,  if 
Quiz  needed 


Test 


Intro  Reps 


□  ■  ■ 

Retrain,  if 

Quiz  Test 


Can  solve  a  problem  in  a  single  context,  maybe. 


Diverse  problems.  with  increased  Can  solve  reflated 
difficulty  and  varied  contexts  problems  in  voned  contexts 
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Encourage  Practice  of  New  Knowledge  and  Skills 

(continued) 


To  help  the  learner 
master  a  complex 
problem,  start  with  fewer 
conditions  and  less 
complexity.  Error 
detection,  hints,  and 
corrections  can  be 
provided.  The  trainer  or 
media  may  intervene  or 
guide  the  learner. 


As  simple  problems  are  mastered, 
instructor/media  support  should  be 
reduced,  and  the  complexity  and/or 
conditions  of  problems  increased. 

Retraining  on  certain  aspects  of 
the  task  may  be  required. 


In  simple  Army  training  terms,  employ: 


Through  a  progression  of 
increasingly  complex  problems, 
learner  skills  and  confidence 
increase.  As  greater  complexity 
is  introduced,  instructor/media 
scaffolding  is  withdrawn. 


Crawl  Walk  Run 
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Provide  Content  in  Logical  Chunks 


This  principle  focuses  on  packaging 
content  into  "bite  size”  pieces  in  a 
framework  and  order  that  will  be 
easier  for  the  user  to  assimilate  and 
learn.4  The  principle  employs 
techniques  that  will: 

•  Foster  and  manage  essential 
processing  -  avoid  overloading 
the  learner.  Focus  the  learner's 
cognitive  and  mental  capacities 
on  building  a  coherent  mental 
representation. 


•  Reduce  extraneous  processing 

-  eliminate  or  minimize 
distractions  and  extraneous 
processing  that  waste  cognitive 
capacity  and  detract  from  the 
learner's  ability  to  build  an 
appropriate  cognitive 
representation. 


Chunked  Content  -  provides  learner 
with  structure,  but  also  supports 
flexibility.  It  eases  navigation  and 
provides  high-level  structure  to 
support  ease  of  understanding  and 
assimilation. 


Chunking  content  is  a  great  start.  However,  learning 
efficiency  and  effectiveness  are  further  enhanced  when 
techniques  are  incorporated  that  manage  essential 
processing  and  reduce  extraneous  processing. 
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Provide  Content  in  Logical  Chunks  -  Manage 
Essential  Processing 

A  learner  can  quickly  be  overwhelmed  by  the  complexity  of  new  ideas, 
knowledge,  and  skills: 

•  Short  term  memory  can  become  overloaded. 

•  New  knowledge  may  not  relate  to  or  associate  with  existing 
knowledge. 

•  Misconceptions  may  develop. 

•  The  number  of  elements  of  new  information  and  the  relationships 
between  them  may  not  be  apparent. 

Cognitive  overload  causes  confusion  and  frustration.  It  can  disrupt  the 
learner's  ability  to  build  a  coherent  mental  representation.  However, 
the  mission  of  your  CBT  is  to  train  the  student;  you  can't  just  "dumb 
down”  the  materials,  delete,  or  omit  complex  pieces  to  make  learning 
easy.  You  can  employ  techniques  and  principles  that  help  the  student 
to  better  cope  with  and/or  control  the  learning  process. 

There  are  three  principles  associated  with  Managing  Essential 
Processing.4  These  principles  reduce  or  eliminate  the  potential  for 
cognitive  overload.  They  are: 

■  Segmenting  -  Present  the  animation  and  material  not  only  in 
chunks,  but  in  learner-paced  segments.  Provide  a  means  to 
control  the  pace  of  presentation. 

■  Pre-training  -  Provide  training  on  essential  prerequisite 
knowledge  and  skills  required  in  primary  training  lessons  and 
modules,  for  example,  names,  locations,  and  characteristics  of  key 
components. 

■  Modality  -  Present  key  ideas  in  spoken  text,  narration  or  dialog  - 
something  for  the  learner  to  hear,  rather  than  merely  written  text 
for  the  learner  to  read. 
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Manage  Essential  Processing  -  Segmenting 

Segmenting  is  similar  to  preparing  to  eat  a  watermelon.  It  makes  less  of  a  mess  and 
is  easier  to  consume  and  enjoy  if  it  is  cut  into  small  pieces. 

Continuous  presentation  (automatic  navigation  from  frame  to  frame)  may  be  appropriate  for  some 
introductions  or  the  lead-in  for  a  scenario.  However,  for  the  presentation  of  training,  "learner-controlled” 
navigation  and  smaller  chunks  of  multimedia  instructional  material  should  be  the  rule. 

Training  research  focused  on  training  complex  subjects  indicates  that  dividing  the  presentation  into  +/- 
10-second  segments  (animation  with  narration)  proved  effective.5  The  effectiveness  was  further 
enhanced  by  providing  the  learner  with  control  of  presentation  rates  and  the  chance  to  repeat  segments 
through  NEXT  and  BACK  navigation  buttons. 

These  features  most  benefit  students  with  low  working  memory  capacity.  Additionally,  they  provide  high 
achieving  learners  with  the  ability  to  rapidly  advance.  All  learners  can  progress  at  their  own  pace. 
Students  can  go  BACK  if  they  didn't  quite  get  it  or  select  NEXT  and  advance  at  a  more  rapid  pace  if  they 
grasp  the  materials.  Offering  a  page  or  narration  REPLAY  feature  also  benefits  new  learners  and  those 
with  low  working  memory. 


=  a  10-second  segment  of 
animation  with  narration 
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Manage  Essential  Processing  -  Pre-training 


Students  learn  more  deeply  from  multimedia  when  they  have  obtained  the  necessary  foundation  knowledge 
and  skills.5  This  foundation  or  prerequisite  knowledge  and  skill  may  include  such  things  as: 


•  The  names  and  characteristics  of  main  concepts 

•  The  names,  locations,  and  characteristics  of  key  components  in  a  system 

•  Tools  to  be  used  when  performing  the  task 

•  Map  reading  or  land  navigation  skills 

Pre-training  is  intended  to  manage  essential  mental  processing  during  the  course  of  training.  For  example, 
students  already  familiar  with  the  names,  locations,  and  functions  or  behaviors  of  components  can  devote 
more  of  their  cognitive  capacity  to  building  a  coherent  mental  representation  or  a  cause- and- effect  model 
of  a  system. 

For  example,  training  on  troubleshooting  a  hydraulic  brake  system  would  be  enhanced  by  Pre-training  on  the 
names,  locations,  and  characteristics  of  the  key  components  of  a  brake  system. 


Components  of  an  Automobile  Hydraulic  Brake  System  Troubleshooting  a  Hydraulic  Brake  System 


A- 24 


Manage  Essential  Processing  -  Modality 


Students  learn  better  from  graphics  with  narrations  and  graphics  or  animations 
rather  than  graphics  or  animations  with  printed  text.4 

•  When  graphics  and  animation  are  paired  with  on  screen  text,  learners 
experience  split  attention.  When  they  view  the  graphics  or  animation 
they  cannot  look  at  and  read  the  text.  When  they  read  the  text  they 
cannot  view  the  graphics  or  animation.  Trying  to  view  both  causes 
incoming  essential  information  to  overload  the  learner's  visual  channel. 

Pairing  animation  and  graphics  with  narration  offloads  the  processing  of 
spoken  words  (narration)  from  the  visual  channel  to  the  verbal  channel. 

Split  attention  also  occurs  when  the  wording  of  narrations  and  necessary 
on-screen  text  have  only  slight  variations.  Vary  the  necessary  on-screen 
text  from  narrations. 


Context  Notes: 

Create  multimedia 
instruction  -  not  a  book  in 
a  box. 

Some  on-screen  text  may 
be  needed  to: 

•  Reinforce  a  key  point 

•  Provide  safety  cautions  or  hazard  warnings 
However,  DO  NOT  use  extensive  on-screen  text  as  a  substitute 
for  pre-training,  additional  media  (more  pages  or  frames),  or 
when  offline  supplemental  reading,  references,  or  exercises 
would  provide  a  more  appropriate  learning  opportunity. 


See  an  example  on  the  next  slide. 


Manage  Essential  Processing  -  Modality 

(continued) 


This  video  (inactivated)  provides  an  example  that  could  easily  transition  into 
CBT.  Narrations  and  animations  are  very  sparsely  supplemented  by  on¬ 
screen  text.  Narrations  and  animation  provide  the  primary  instructional 
content. 


Provide  Content  in  Logical  Chunks  -  Reduce  Extraneous  Processing 

Since  a  learner  can  quickly  be  overwhelmed  by  the  complexity  of  new  ideas,  multiple 
steps,  knowledge,  and  skills,  it  is  critical  to  not  only  manage  essential  processing, 
but  also  to  guard  it  by  reducing  or  eliminating  extraneous  processing.  This  can  be 
accomplished  by  keeping  the  learner's  attention  (eyes  and  ears)  focused  through 
effective  learning  techniques,  and  by: 

•  Limiting  or  eliminating  distractions  or  unnecessary  details  that  are  a 
waste  of  cognitive  capacity. 

•  Minimizing  the  disruptions  of  the  learner's  effort  to  build  an 
appropriate  cognitive  representation. 

Cognitive  overload  causes  confusion  and  frustration.  Your  media  design  should  employ 
techniques  that  help  the  student  to  better  cope  with  and/or  control  the  learning 
process.  There  are  five  principles  associated  with  Reducing  Extraneous 
Processing.4 

These  principles  reduce  or  eliminate  the  potential  for  cognitive  overload. 

•  Coherence  -  avoid  insertion  of  materials  that  are  not  essential  to 
building  the  learner's  mental  model. 

•  Signaling  -  draw  attention  to  essential  materials.  This  can  be  done  with 
narration  emphasis,  highlighting,  and  drawing  attention  to  essential  on-screen  text. 

•  Redundancy  -  avoid  duplication  in  narration  and  on-screen  text.  Duplication 
may  contribute  to  cognitive  overload. 

•  Spatial  contiguity  -  provide  a  logical  layout  and  organization  on  pages  and 
frames  of  material.  Students  learn  better  when  corresponding  words  and  pictures 
are  presented  near  rather  than  far  from  each  other  on  the  page  or  screen. 

•  Temporal  contiguity  -  present  narrations  synchronized  with  animations. 
Students  learn  best  when  actions  in  animations  and  narrations  are  in  step  and 
narrations  address  ongoing  actions;  present  graphics  and  corresponding  narrations 
at  the  same  time. 
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Reduce  Extraneous  Processing -Coherence 


Students  are  better  able  to  focus  their  attention  and  capacities  on  essential  material 
when  extraneous  materials  that  could  distract  them  are  eliminated.4  They  learn  more 
deeply  when  extraneous  materials  are  excluded.  The  page  design  and  supporting 
graphics  should  focus  on  key  learning  objectives.  Avoid  the  temptation  to  introduce 
remotely  related,  unnecessary,  or  "nice  to  know”  information  in  any  form:  animations, 
video,  text,  or  images. 


Coherence  Examples 


Extraneous  materials  dpi  the  screen  have  an 
Influ  ence  simitar  tf>  text  in  g  while  driving. 
However,  instead  of  "distracted  driving"  the 
result  Is  diverted  attention  and  disrupted 
learning. 


Symbols  and  may  contribute  to 

learning.  If  understood  by  the  learning  audience. 
Ef  not  understood,  they  can  be  terrible 
distractions , 


Would  your  training  andtiGece  understand  these 
without  explanation? 


Simple  grdfihitt  Idr  pointing  provide  l^g 
distraction  and  better  accuracy  than  finders  or 
human 


Key  Conneclicfi 


Clean  diagrams  and  graphics  are  easier  for  the  new 
[earner's  eyes  to  navigate  than  cluttered  photos. 
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[Grab  your  reader's  attention  with  a  great  quote  from  the  document  or  use 
this  space  to  emphasize  a  key  point.  To  place  this  text  box  anywhere  on  the 
page,  just  drag  it.] 


A-29 


Reduce  Extraneous  Processing  -  Signaling 


Students  are  better  able  to  focus  their  attention  and  capacities  on  key  or 
essential  learning  material  when  it  is  emphasized.  Signaling  may  be 
achieved  through: 


•  Narration - Tell'em  that  it  is  important. 

or  highlighting 


Illumination 


Underlining,  bolding,  or  italics  (or  a  combination) 


•  COLORS, 


Flashing  Text 


or  WARNINGS 


Methods  that  attract  and  focus  the 


However,  focus  the 
learner's  attention,  but 
Don't  Over  Do  It! 
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Reduce  Extraneous  Processing  -  Signaling  (continued) 

More  subtle  techniques  (fewer  extraneous  distractions)  will  result  in  better  success.  Use  framed 
text  boxes  and  chose  fonts  that  focus  the  learner's  attention.6  Be  consistent  with  signaling 
techniques;  learners  "catch  on"  and  are  alert  to  techniques  consistently  used.  The  insert  below 
provides  details  for  using  the  simple  method  of  font  selection  for  signaling: 

Fonts  are  sometimes  selected  for  use  in  multimedia  designs  for  their  readability  or  esthetic  qualities. 
For  example,  Arial,  Verdana,  or  Helvetica  are  neat,  defined,  and  clean  san  serif  fonts.  These  fonts 
are  frequently  selected  for  titles,  short  bullets,  and  captions  (on-screen  text).  They  are  ideally  used 
when  there  is  not  too  much  text  and  readability  is  not  an  issue. 

However,  studies  indicate  that  font  selection  has  a  significant  impact  on  student's  ability  to  recall 
information.  Serif  type  fonts,  those  with  a  finishing  mark,  embellishment,  or  a  line  at  the  end  of  a 
stroke  in  a  letter  or  symbol,  can  improve  recall  by  approximately  9%.  This  improvement  has 
practical  value.  On-screen  text  in  a  serif  type  font  can  be  used  to  support  the  conveyance  of  key 
teaching  points  or  to  emphasize  safety  information  when  training  new  skills  or  knowledge. 

What  accounts  for  this  increase  in  recall  when  using  serif  fonts?  While  san  serif  typefaces  are  clean 
and  bold,  they  are  slightly  less  legible  than  serif  faces.  The  serif  fonts  seem  to  make  rows  of  text 
appear  to  set  upon  a  line.  These  more  distinct  rows  of  text  may  be  perceptually  easier  to  separate. 
This  same  effect  is  often  achieved  when  a  reader  uses  a  straightedge,  underlines,  or  highlights  a 
section  of  text.  These  techniques  more  clearly  separate  the  target  text  from  the  surrounding  text 
and  information. 

More  easily  perceiving  a  row  of  text  makes  the  text  easier  to  read.  Less  attentional  resources  are 
required  for  the  process  of  reading  and  more  attentional  resources  can  then  be  devoted  to  attending 
to  the  message  or  information  being  conveyed.  This  may  result  in  deeper  processing  and  an  easier 
recall  of  the  information. 


AaB b C c  Saus se,inbiu 
AaBbCc  saiffout 
AaBbCc  — 


Font 

Type 

Size 

Example 

Times  New  Roman 

serif 

14 

One;  two.  three 

Palatino 

serif 

14 

One.  two.  three 

Courier 

seri  f 

14 

One,  two,  three 

Monaco 

san  serif 

14 

One,  two,  three 

Verdana 

san  serif 

14 

One,  two,  three 

Helvetica 

san  serif 

14 

One,  two,  three 

Arial 

san  serif 

14 

One,  two,  three 
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Reduce  Extraneous  Processing  -  Redundancy 


Displaying  the  narration  in  on-screen  text,  potentially  increases  cognitive 
overload.  Learners  are  better  able  to  focus  their  attention  and  capacities 
on  key  material  or  essential  learning  when  the  incoming  verbal  materials 
are  reduced  to  a  single  stream.4- 5  While  it  may  seem  that  providing  both 
a  narration  and  parallel  on-screen  text  would  be  helpful  since  some 
learners  prefer  to  read  and  some  prefer  to  listen,  the  opposite  is  true. 
Adding  on-screen  text  creates  extraneous  processing.  The  learner  tries  to 
reconcile  the  two  incoming  verbal  streams  reducing  cognitive  capacity  and 
deep  learning. 

Images  or  animation  and  narration  alone  prove  more  effective  in  learning. 
However,  when  some  on-screen  text  is  required  for  clarification,  additional 
details,  or  emphasis,  consider: 

•  Adding  media  -  another  frame  or  two.  .  .  or  three 

•  Assure  dissimilarity  of  on-screen  text  and  narrations 

•  Use  short  bullets  (not  more  than  3-4  concise  phrases) 

•  Phase  in  the  on-screen  text  -  -  after  the  narration  and  animation 


(NOTE:  The  short  list  of  bullets  above  is  an  example.) 
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Reduce  Extraneous  Processing -Spatial  Contiguity 


Students  learn  better  when  corresponding  words  (on-screen  text)  and  pictures  are  presented  near  rather 
than  far  from  each  other  on  the  page  or  screen.4 

Spatial  Contiguity  applies  to  captions,  labels,  and  explanatory  text  associated  with  pictures,  images,  and 
graphics.  While  the  urogenital  systems  of  frogs  may  never  be  the  focus  of  your  CBT,  these  contrasting 
images  and  labeling  schemes  provide  examples. 


BAO 
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Reduce  Extraneous  Processing -Temporal  Contiguity 


Students  learn  best  when  actions  in  animations  and  narrations  are 
synchronized.4  Present  graphics  and  corresponding  narrations  at  the 
same  time  and  synchronize  the  descriptive  words  with  the  actions  that 
the  learners  are  viewing. 

Temporal  Contiguity  applies  to  synchronization  of  narrations  with 
animations  or  videos.  Most  of  us  have  watched  a  "B  grade"  Kung  Fu 
movie  or  foreign  made  Western.  These  low  budget  movies  have  dubbed 
English  dialog  and  sound  effects  over  the  original  sound  track.  Some 
display  continuous  subtitles  during  the  foreign  dialog.  Occasionally  the 
actor's  lips  keep  moving  after  a  sentence  ends,  or  they  stop  moving 
while  the  overdubbed  dialog  continues.  Sometimes  the  slap  of  a  sharp 
blow  occurs  before  or  after  your  eyes  tell  you  that  the  blow  is  landed. 

While  humorous  in  the  "B  grade"  action  movie,  animations  and  narrations 
in  multimedia  instruction  delivered  out  of  synchronization  or 
disconnected  have  a  negative  impact  on  the  learner.  The  result  of  this 
visual  and  verbal  disconnect  is  often  misconception  or  cognitive  overload 
as  the  learner's  cognitive  capabilities  busy  themselves  trying  to  make 
sense  of  the  disjointed  presentation. 


A-34 


Provide  a  Meaningful  Learning  Experience  -  Promote 
Generative  Processing 


Knowledge  construction  relies  on  active  mental  processing  of  perceptions, 
what  the  learner  sees  and  hears.  This  active  mental  processing  is 
commonly  referred  to  as  Generative  Processing.  Its  end  result  is 
understanding.  If  your  instructional  design  and  presentation  techniques 
have  been  successful  in  Reducing  Extraneous  Processing  (slide  20)  and 
Managing  Essential  Processing  (slide  25),  you  have  freed  up  the 
learner's  cognitive  capacity.  There  is  now  cognitive  capacity  for 
Generative  Processing,  mental  processing  potential  for  knowledge 
construction  and  understanding  -  real  learning.4-7 

However,  this  doesn't  just  happen.  Your  design  should  facilitate  the 
learner's  engagement  in  Generative  Processing,  otherwise  some  or 
all  of  this  available  cognitive  capacity  will  remain  idle.  Techniques  that 
promote  Generative  Processing  include: 

•  Multimedia  -  present  learning  materials  in  words  and  pictures  (graphics, 
images,  video,  and  animation),  not  just  words  alone. 

•  Personalization  -  present  words  and  narrations  in  a  conversational  style 
rather  than  formal  style.  Your  narrator  should  develop  a  "social 
partnership"  with  the  media  in  talking  to,  not  at  the  learner. 

•  Voice  -  narrations  should  be  vocalized  in  a  manner  that  is  human  and 
avoids  the  impression  of  machine  or  synthetic  voice. 

•  Embodiment  -  on-screen  agents  (characters,  images,  and  animations) 
aid  in  the  learning  process  when  they  display  human-like  gestures 
(movement,  eye  contact,  and  expressions).  Human- like  actions  create  a 
sense  of  social  presence  that  learners  relate  with  and  react  positively 
toward. 

•  Image  -  static  images  of  a  speaker  (e.g.  narrator)  on  the  screen  may 
cause  a  distraction. 
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Promote  Generative  Processing  -  Multimedia 


CBT  should  employ  the  power  of 
multiple  forms  of  media  (graphics, 
images,  video,  audio,  animation,  and 
text)  to  engage  the  learner  through 
sight  and  sound  and  support  the 
learner's  active  mental  processing  of 
the  perceptions  and  information 
conveyed  by  the  media.4  While  the 
learner  may  be  limited  to  some  extent 
by  physical  interfaces,  the  array  of 
media  and  interactive  programming 
can  bring  the  learner  into  a  near 
virtual  environment.  To  reduce  this 
experience  to  words  alone  is  an 
injustice.  Without  graphics  and  other 
media,  CBT  is  just  a  "book  in  a  box". 

With  inputs  from  the  verbal  channel, 
limited  generative  processing  is 
possible.  Cognitive  capacities  are  idle 
while  learning  and  understanding 
become  more  difficult. 

Example:  You  are  on  a  mountain 
road  in  East  Tennessee  near 
Richardson  Cove  crossroads.  It  is 
getting  dark,  a  storm  is  coming,  and 
you  need  to  get  to  your  destination  in 
Pine  Grove,  a  few  miles  away.  You 
query  your  smart  phone  for 
directions,  but  all  that  you  can  get  are 
written  directions.  Good  luck. 


Start  out  going  west  on  Richardson  Cove 
Rd  toward  Maples  Branch  Rd 

0.3  mi 

Take  the  1st  left  onto  Maples  Branch  Rd. 

0.05  mi 

Turn  right  onto  Pittman  Center  RdfTN- 
416. 

1.5  mi 

Turn  left  onto  Birds  Creek  RdfTN454. 

0.1  mi 

Take  the  1st  right  onto  Upper  Middle 
Creek  Rd 

5.4  mi 

Upper  Middle  Creek  Rd  becomes 

Dolly  wood  Ln. 

0.7  mi 

Turn  left  onto  Dollywood  Lnn"N449. 

0.3  mi 

Turn  right  onto  Parkway(US441  N)US- 
321  5/TN-71.  Continue  to  follow 

ParkwayWS441  N/TN-71. 

2.9  mi 

Turn  right  onto  Showplace  Blvd 

0.01  mi 

Welcome  to  PINE  GROVE,  TN 

The  driver's 
view 

through  the 

windshield 

of  a  crooked 

Smokey 

Mountain 

road. 


(View  the  details  then  advance  for 
more). 
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Promote  Generative  Processing  -  Multimedia 

(continued) 


You  try  again.  This  time  the  graphics  come  with  the  words. 


1 .  Start  out  going  west  on  Richardson  Cove 
0  Rd  toward  Maples  Branch  Rd 


2.  Take  the  1st  left  onto  Maples  Branch  Rd 

If  you  are  on  Maples  Branch  Rd  and  reach  Laurel 
^  I  Ln  you've  gone  about  0.4  miles  too  far 


' 


3.  Turn  right  onto  Pittman  Center  RdfTN- 
f*  416 


4  Turn  left  onto  Birds  Creek  Rd/TN-454 


5.  Take  the  1st  right  onto  Upper  Middle 
Creek  Rd 

fF  If  you  reach  Catons  Chapel  Rd  you've  gone  about 

0.3  miles  too  far 


6.  Upper  Middle  Creek  Rd  becomes 
^  Dollywood  Ln 


Graphics  and  words  (synchronized  information  on  the  visual 


^  7.  Turn  left  onto  Dollywood  LnfTN-449 


and  verbal  channels)  have  a  very  positive  impact  on 
understanding. 


&> 


Turn  right  onto  Parkway/US-441  N/US- 
321  S/TN-71.  Continue  to  follow 

Parkway/US-441  N/TN-71 

Parkway  is  0.1  miles  past  S  River  Rd 
Pizza  Rut  is  on  the  comer 


You  just  might  make  it  from  Richardson  Cove  to  Pine 
Grove  now. 


9. 


r* 


Turn  right  onto  Showplace  Blvd 

Showplace  Blvd  is  0. 1  miles  past  Sugar  Hollow  Rd 
Hampton  Inn  &  Suites  Pigeon  Forge  On  The 
Parkway  is  on  the  comer 
If  you  reach  Plaza  Dr  you've  gone  a  little  too  far 


J  j  ,Ve  curie  :o  PINE  GROVE.  TN 


0.3  mi 

0.05  mi 

1.5  mi 

0.1  mi 

5.4  mi 

0.7  mi 

0.3  mi 

2.9  mi 

0.01  mi 
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Promote  Generative  Processing  -  Personalization 


Narrations  and  dialog  should  be  presented  in  a  conversational  rather  than  formal  style.7  This  is  a  bit  more 
than  replacing  "the"  with  "your",  it  may  require  some  thought.  However,  research  shows  that  when 
conversational  or  "polite"  wording  is  used,  the  learner  develops  a  sense  of  social  partnership  with  the 
narrator.  Within  that  "social  partnership"  the  learner  tries  harder  to  make  sense  of  what  his 
conversational  partner  is  saying  and  trying  to  convey.  Learners  process  the  key  points  of  the  dialogue 
and  the  result  is  a  better  transfer  of  knowledge  and  skills  from  the  media.  The  two  narration  examples 
below  provide  parallel  narratives  on  a  procedure  of  watch  repair.  They  provide  a  contrast  of 
conversational  vs.  formal  styles. 


Conversational  or  "Polite'1 


Formal 


now  tnai  you  have  opened  (he  back 
and  accessed  the  ‘guts'  or  movement 
of  the  watch,  your  next  step  is  ‘Celling 
down*  the  mainspring  The  mainspring 
is  a  long  piece  of  steel  It  is  rolled  up 
and  held  tightly  mside  a  cylinder  called 
the  “mainspring  barreT.  in  this  state 
the  mainspring  is  under  constant 
pressure,  it  is  a  rigid  ribbon  that  wishes 
to  lay  flat  and  net  be  rolled  up.  When 
you  wind  a  watch,  the  mainspring  rolls 
even  light  or,  storing  the  energy  needed 
to  run  the  watch.  Your  task  is  to  make 
sure  that  the  watch  mainspring  is  not 
under  pressure.  You  do  this  by  ‘letting 
down'  or  unwinding  the  mainspring. 
Taking  a  wound  watch  apart  could  not 
only  damage  the  watch  but  may  send 
pieces  flying  around  your  work  area. 
The  proper  way  to  ‘let  down"  the 
mainspring  jj  a  bit  mcky  First,  wind 
the  watch  a  bit  so  you  can  observe  the 
moving  parts  and  locale  the  ‘click".  The 
*d fck*  is  a  (my  melai  stopper  that  clicks 
every  time  that  it  passes  over  a  gear. 


Identify  and  disengage 
tihe'diek"  to  let  down 
the  mainspring  of  a 
watch. 


1 

Better  style 
for  CBT  narrations 

Okay  for 
written 
technical 
dFrections 


The  watch  back  is  now  removed  and 
the  movement  or  “guis-  can  be 
accessed.  The  next  slep  is ‘letting 
down"  Ihe  mainspring.  The  mainspring 
is  a  long,  flat  ribbon  of  steel-  It  is  eighily 
coiled  under  pressure  and  contained 
inside  a  cylinder  called  the  'mainspring 
barrel'  .  When  a  watch  is  wound,  the 
mainspnng  is  rolled  tighter,  storing  Ihe 
energy  needed  to  run  ihe  welch 
Before  proceeding  further,  ihe  energy 
must  be  released  from  the  mainspnng. 
This  is  accomplished  by  unwinding  ihe 
watch  in  a  process  called  ‘letiing  down* 
Ihe  mainspring.  The  mainspring  must 
be  ‘lei  down"  to  avoid  damage  to  Ihe 
watch.  This  damage  can  sometimes  be 
calamitous  with  delicate  parts  breaking 
and  delachmg  from  the  moveimenl 
Therefore,  the  process  of  "Idling  down' 
che  mainspring  is  deticate  and  comptex. 
The  process  begins  by  slightly  winding 
the  watch  while  observing  Ihe 
movement  to  locate  Ihe  small  metal 
siopper  or ‘click*.  This  small  stopper 
emits  a  'click*  sound  every  time  lhat  it 
passes  Over  a  gear, 
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Promote  Generative  Processing  -  Voice 


Training  research  indicates  that  human  voice  narrations  and 
dialog  promote  generative  processing  in  learners  better  than 
machine  synthesized  (text-to-speech)  voice.7  The  human  voice 
with  its  tone,  inflection,  and  familiar  qualities  elicits  a  sense  of 
social  familiarity  in  learners.  While  this  is  no  doubt  true,  there  are 
some  questions  that  should  be  carefully  weighed  when  producing 
CBT  on  a  constrained  budget:8 

•  Why  consider  machine  synthesized  voice  -  text-to-speech 
(TTS)? 

•  Why  not  use  human  narration? 

•  What  are  some  considerations  for  just  using  on-screen  text? 

•  How  will  the  narration  affect  the  quality  of  the  learning?  How 
will  students  accept  the  voices? 

Why  Use  Text-to-Speech? 

•  It  is  the  cheapest  and  fastest  method  to  produce  narration;  it 
can  be  done  in-house  with  commercially  available  software  (a 
one-time  expense). 

•  The  voice  quality  and  variety  of  TTS  has  improved  in  recent 
years;  it  is  widely  used  and  employed  in  the  training  and 
information  programs  of  major  corporations.  It  has  matured 
beyond  the  "mechanized  sound"  and  "robotic  cadence";  TTS 
voice  can  be  "trained". 

•  It  is  easily  integrated  into  CBT  with  minimal  programming. 

•  It  is  the  easiest  method  of  narration  to  revise  on-the-fly  and 
update;  the  software  permits  quick  edit  of  voice  speed,  single 
words,  and  spellings  can  be  modified  (trial  and  error  method) 
to  incorporate  military  jargon  and  pronunciation  of  acronyms. 

•  A  computerized  voice  is  generally  consistent  in  pronunciations 
and  speaks  at  a  steady  pace.  These  qualities  allow  most 
learners  to  understand  the  material  more  easily  (especially 
when  English  is  not  the  student's  primary  language). 

•  It  provides  a  quick  and  easy  means  to  listen  to  and  assess 
scripts. 

•  Some  recent  research  indicates  that  TTS  is  "good  enough";  it 
is  accepted  by  students,  and  does  not  detract  from  learning. 
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Promote  Generative  Processing  -  Voice 

(continued) 


Why  not  use  Human  Narration? 

■  Live  voice  narrations  are  expensive  and  time-consuming 
to  produce;  they  require  specialized  equipment,  facilities, 
and  talent.  Requirements  are  frequently  out-sourced. 

■  They  are  difficult  to  produce  since  only  a  few  people  have 
the  voice  quality  and  minimal  accents  to  perform  the 
recordings.  This  creates  an  initial  resource  constraint  for 
the  creation  of  material. 

■  Live  narrations  are  significantly  more  complicated  and 
costly  to  update;  they  require  the  use  of  the  original 
voice  talent  for  most  minor  edits  or  when  changing  large 
sections  of  narration. 

■  Most  raw  recording  requires  some  minor  edits.  Unedited 
recordings  sometimes  retain  pacing  issues  (too  fast  or 
too  slow),  sound  quality  issues,  incorrect  pronunciation, 
or  improper  voice  emphasis.  Corrections  could  require 
redoing  an  entire  narration  sentence  or  segment. 

Why  not  just  use  on-screen  text? 

•  Most  learners  have  much  better  vocal/ listening 
comprehension  than  reading  comprehension  skills. 

•  Audio  contributes  to  more  effective  learner  engagement. 

•  Large  amounts  of  on-screen  text  would  be  required  to 
support  instructional  graphics  or  animations;  narration  better 
augments  graphic  materials  and  enhances  learning. 

•  Text  limits  the  opportunity  for  human  emotion  and  voice 
emphasis. 

•  Text  alone  can  be  very  boring  and  mundane. 
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Promote  Generative  Processing  -  Voice 

(continued) 


Summary 

•  TTS  is  the  cheapest  and  fastest  method  to  produce 
narrations  and  to  make  revisions/updates. 

•  When  TTS  is  used,  including  a  caveat  at  start  of  training 
may  help  to  set  student  expectation  about  computer  voice. 

•  When  TTS  is  used  as  the  primary  source  of  narrations,  selected 
segments,  on-screen  characters,  and  scenarios  that  benefit 
from  "emotion"  and  voice  inflection  can  be  developed  in  live 
voice  to  enhance  learning  objectives  while  reducing  expense. 

•  However,  consider  methods  and  techniques  that  best  address  the 
needs  of  your  learners: 

o  Learners  who  derive  more  from  reading  than  listening 

sometimes  advocated  for  a  narration  mute  button  with  script 
available  on-screen.  This  enhances  the  learning  of  those  who 
prefer  that  modality. 

o  Narration  with  pop-up  on-screen  text  may  be  preferred  by  those 
for  whom  English  is  a  second  language.  Some  learners  in  this 
category  have  indicated  that  this  technique  helped  them  learn 
English. 

Examples  are  provided  on  the  next  slide. 
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Promale  Generative  Processing  —  Vote 
EXAMPLES 

For  your  comparison,  variations  with  narration  texts  are  provided  - 


‘"Breathy,  conversational" 

tts 


"Dialog'  Live  Voice 


"Complex  vocabulary" 

TTS 


In  my  younger  days,  the 
older  sergeants  told  me 
that  if  the  Army  wanted 
me  Iq  have  an  attitude, 
they  would  have  issued 
me  one.  However, 
altitudes  are  a  fact  of 
nature.  AS  NCOS  end 
iead&rsr  we  know  it  is 
much  easrer  to  motivate 
Soldiers  to  do  things  that 
they  enjoy  and  under¬ 
stand.  ie  fs  always  more 
difficult  if  they  don't  enjoy 
the  task,  or  (Pause)  they 
feel  pressured.  In  this 
example.  Sergeant  Golden 
is  meeting  with  his  Platoon 
Sergeant.  Listen  in;  the 
news  is  not  all  good. 
Sergeant  Golden's  "to  do 
list  is  about  to  get  longer. 


sfc  ivy:  sgt  Golden,  ]  have 
some  good  news  and  some  had 
newEj  which  do  you  want  a 
first? 

SGT  Golden:  Just  give  it  to  me 
straight  Sergeant  Ivy. 

SFC  ivy:  Okay.  The  First 
Sergeant  was  just  alerted  that 
your  squad  leader.  Staff 
Sergeant  Black,  is  being 
reassigned  to  the  old  Guard. 

He  will  be  clearing  within  a 
week-  Your  squad  has  been 
tops  in  the  Company  maybe 
the  Battalion,  You  are  junior 
but  you  have  played  a  major 
role  in  that  good  performance. 
You  have  demonstrated  a  lot  ot 
potential.  The  Platoon  leader 
and  I  thought  that  werd  just 
move  you  up,  but  the  First 
Sergeant  says  that  there  are 
two  more  E  (say  the  letter 
fives  in  the  Company  w'ho 
deserve  a  chancy  Marsh  in  3rd 
Platoon  and  Wilson  in  2nd 
Platoon. 

SGT  Golden:  {in  a  frustrated 
tone)  Come  on  Sergeant  Ivy; 
isn't  Wilson  the  guy  from  2nd 
Platoon  who  is  al ways  ch ecki ng 
out  early.  Doesn't  he  have  kids 
who  are  always  getting  sick  at 
school  or  his  wife  £snrt  supposed 
to  drive  or  something? 

PARTIAL  TEXT  _  .  . 


For  example,  if  your  self- 
learntng  includes  a 
structured  course, 
(PAUSE  1)  the  required 
learning  resources  may 
be  provided  to  or 
identified  for  you. 
Additionally,  choosing 
supplemental  resources 
may  be  guided  by  the 
instructor  or  derived 
from  course  materials. 
For  military  subjects r  the 
Army  has  invested 
heavily  in  creating 
courses  that  guide 
Soldiers  toward  success. 
However.,  (pause  2]  if 
your  learning  approach 
is  more  self-directed  or 
independent,  narrowing 
down  and  selecting 
learning  resources  may 
prove  a  bit  more 
challenging.  Let  us 
examine  some 
techniques  that  may 
help  you  assess  available 
resources  and  determine 
those  that  may  best  suit 
your  learning  needs. 
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Promote  Generative  Processing  -  Embodiment  (Human  Likeness) 

Images  and  photos  have  a  positive  impact  on  the  learner  when 
correctly  used.  On-screen  agents  (images  of  people/Soldiers  and 
animated  characters)  greatly  aid  in  the  learning  process  when 
they  display  human-like  gestures  (movement,  eye  contact,  and 
expressions).  Training  research  indicates  that  human- like  actions 
create  a  sense  of  social  presence  that  learners  relate  to  and  react 
positively  toward.7 

The  development  of  video  segments  with  actors  (professionals,  volunteers, 
or  Soldiers)  can  be  resource  intensive  or  cost  prohibitive.  However,  the 
broadly  distributed  training  simulation  systems  such  as  Virtual 
Battlespace  (VBS)  offer  multimedia  training  developers  unique  capabilities 
to  create  realistic,  editable  scenarios  and  vignettes.9 These  training 
simulation  systems  were  designed  to  provide  immersive,  real-time,  multi¬ 
user,  exercises  mimicking  real-world  situations.  They  have  extensive 
content  libraries  providing  personnel  (military  and  civilian),  equipment, 
static  objects,  and  simulated  effects  to  populate  varied,  editable  scenarios. 

For  example,  VBS3  features  include  U.S.  Soldiers  from  all  Services  as  well 
as  11  foreign  militaries.  It  provides  more  than  200  civilian  avatars  of 
varied  ethnicities,  genders,  and  nationalities  with  realistic  facial  features. 
Content  libraries  include  military  and  civilian  models  of  nearly  300  tracked 
vehicles,  1,000  wheeled  vehicles,  more  than  100  planes  and  unmanned  air 
systems,  and  over  550  small  arms  and  weapons.  The  scenarios,  terrain, 
opposing  force,  and  environmental  circumstances  can  be  varied  to  create 
the  desired  conditions  to  complement  scenarios.  Some  facial  expressions 
and  gestures  can  be  programmed  into  avatars  and  some  models  can  be 
selected  or  programmed  to  act,  react,  or  display  actions  associated  with 
emotions  and  behaviors. 

Scenarios  and  scenes  can  be  converted  to  video  and  still  photos  using  a 
variety  of  commercial  image  capture  software  suites  for  use  in  CBT.  In 
short,  you  can  use  VBS  systems  to  create  and  capture  realistic  still  imagery 
and  video  to  enhance  your  CBT. 

(See  the  next  slide  for  VBS  model  and  avatar  examples) 
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Promote  Generative  Processing  -  Embodiment 

(continued) 
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Promote  Generative  Processing  -  Embodiment  WARNING 

(continued) 


Take  care  to  choose  images  and  video  that 
authentically  portray  the  correct  uniforms  and 
equipment.  Soldiers  quickly  challenge  the  accuracy 
and  credibility  of  CBT  content  when  dated  uniforms, 
outdated  equipment,  or  incorrect  doctrine  or  tactics 
are  used.  Clearly  identify  the  use  of  historic 
examples. 


Woodland 

BDU 

early  1980s  to 
mid-2000s 


UCP  ACU 

2004  to  mid-2018 


OEFCP-OCP  OCPACU 

ACU  issue  begins 

selective  issue*  2015 

2010  to  2016 


(See  the  next  slides  for  video  and  still  image  examples) 
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Promote  Generative  Processing - 
Embodiment 

(continued) 


Images  and  video  segments  may  be  edited  and  assembled  to  support 
your  instructional  design.  Scenario  frameworks  should  provide 
plausible  situations.  These  situations  should  be  ones  that  the  learner 
can  relate  to  with  some  familiarity,  but  they  should  be  presented  with 
some  degree  of  uncertainty  or  unpredictability. 


In  these  examples,  still  images  are  used  to  create  the  illusion  of  a 
conversation  between  two  on-screen  characters.  Note  that  as  the 
scene  changes,  varied  postures  and  perspectives  are  used  to  provide 
the  learner  with  an  illusion  of  human-like  movements.  Voice 
inflections  and  emotions  expressed  in  the  dialog  also  become  key  to 
convey  a  true- to- life  conversation.  A  recognized  mentor  or  guide 
appears  outside  the  scene  to  provide  commentary  and  relate  the 
scenario  to  learning  objectives. 

For  the  video  (inactivated), 
segments  from  varied 
noncontiguous  action 
scenes  have  been  spliced 
together.  This  technique 
saves  time  since  only 
action  scenes  are  shown 
to  the  learner.  Lulls  and 
minor  actions  are  omitted. 

This  technique  keeps  the 
learner  engaged  with  near 
constant  action.  Key 
actions  and  events  are 
presented  in  the  expected 
sequence  of  occurrence,  a 
logical  flow  that  the 
learner  can  relate  to  and 
follow  without  additional 
cues.  The  combination  of 
varied  still  images  and 
video  create  a  sense  of 
social  presence.  The 
learner  can  relate  to  the 
situation  and  the  scenario. 
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Promote  Generative  Processing  - 
Embodiment 

(continued) 


For  the  second  example  (inactivated),  a  segment  of  an  introduction 
is  provided.  Note  the  brief  introduction  by  an  unanimated  "talking- 
head”.  This  transitions  to  a  dialog  between  two  on-  screen  agents 
or  characters.  Varied  still  images  from  different  perspectives  are 
used  instead  of  animation  (movements,  changes  to  facial 
expressions,  and  gestures)  to  create  a  sense  of  social  presence  for 
the  learner.  These  varied  perspectives  keep  the  learner  engaged 
and  are  less  resource  and  talent  intensive  than  animation. 
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Promote  Generative  Processing  -  Image 


Effective  narration  is  essential  to  CBT.  However,  deep  learning  is  not 
influenced  by  seeing  a  narrator  or  talking- head  on  the  screen.7  In 
the  days  of  black  and  white,  three-channel  network  television  (before 
home  computers  and  cell  phones),  viewers  were  held  spellbound  by 
the  nightly  images  of  news  anchors  on  their  screen.  Personalities  like 
Chet  Huntley  and  David  Brinkley  became  icons,  trusted  to  deliver 
accurate  reporting  and  commentary. 

However,  times  and  learner  expectations  have  changed,  the  same  is 
not  true  for  effective  CBT.  While  an  image  or  text  on  the  screen 
during  narration  normally  has  a  positive  impact  on  the  learner,  static 
or  talking- head  images  on  the  screen  may  cause  distraction  that 
detracts  from  any  social  benefit. 

Timing  is  critical.  If  the  talking-head  performs  a  brief  transition,  it 
may  be  okay,  but  when  it  stays  on  the  screen  too  long,  interest 
fades.  Generative  processing  is  enhanced  by  meaningful  images  with 
supporting  narrations. 


Talking-heads 
alone  (animated  or 
human)  are 
distractors  and 
seldom  contribute 
to  learning. 
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Assess  Learning 


Assessments  can  be  used  both  as  a  gauge  of  the  state  or  level  of  learner 
knowledge,  skills,  and  understanding  (summative)  and  as  a  measure  of 
instructional  effectiveness  of  training  methods  and  media  (formative).  The 
same  assessment  tools  can  accomplish  both  purposes  simultaneously. 

When  groups  of  learners  fail  to  perform  or  perform  poorly  on  assessments 
in  selected  skills  or  fail  to  display  the  desired  level  of  knowledge  or 
understanding  of  a  task  or  subject,  the  training  methods  or  media  may  be 
at  fault.  Learner  performance  or  assessment  responses  may  assist  in 
formative  assessment  to  pinpoint  or  diagnose  training/instructional 
shortcomings. 

However,  for  media  and  instructional  methods  that  have  been  validated, 
three  levels  or  types  of  summative  assessments  or  tests  may  prove 
beneficial  to  the  learners: 

-  A  pre-assessment,  before  training  to: 

•  Identify  learners  requiring  pre-training  on  essential 
foundation  skills  or  knowledge 

•  Support  tailoring  of  training  (to  meet  the  learners  where  they  are) 

•  Determine  a  learning  path 

•  Mid-course  assessments  after  chunks  or  modules  to 
determine  learner  progress  and  to  reinforce  enabling 
learning  objectives 

-  An  end  of  training  or  post-training  assessment  to  determine  the 
learner's: 

•  Level  of  proficiency 

•  Attainment  of  standards 

•  Retraining  needs 

These  assessments  may  be  verbal,  hands-on,  paper  based,  or  embedded  in 
the  CBT. 

Assessments  are  also  a  learning  activity.  They  provide  additional  practice. 
However,  to  provide  value,  all  assessments  should  provide  feedback  to  the 
learner. 
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Provide  Assessment  Feedback 


This  example  shows  the  use  of  a  whole  task  evaluation.  This  technique  provides  feedback  to  support 
tailoring  of  initial  training  or  retraining. 

Training  recommendations  are  based  on  the  Assessment. 


1.  Designate/prepare/adjust 
primary  individual/team/ 
crew  fighting  positions  to: 

2.  Designate/prepare: 

-  Alternate  positions 

-  Supplementary 
positions 

(These  positions  should 
meet  the  same  standards 
as  primary  positions.) 

El 

el 

Provide  effective/ 
adequate  fields  of  fire. 

£1 

3.  Employ  fire  control 
measures  effectively 

El 

a  i 

a  [ 

Enhance  Soldier/unit 
survival  and  safety. 

a 

El 

E) 

Assessment  Scoring:  =  GO  =  NO  GO 

If  all  categories  are  a  GO,  congratulations,  you 
have  attained  the  standards  for  this  lesson  and 
may  end  training  or  elect  to  take  selected  topics. 
If  you  elect  to  end  training,  select  NEXT  to  EXIT. 

If  any  elements  of  the  task  was  assessed  as  a 
NO  GO,  the  Recommended  Training  Topic  or 
Topics  have  been  illuminated.  Upon  completion 
of  additional  training  you  win  be  routed  to  the 
^^-T  rai^^^sses^^^ 

Suggested  sections  you  should  review: 


Recommended  Training  Topics 


Platoon  and  Squad  Defensive  Planning 


Select  a  topic  to  continue  training. 


Selecting  Weapon  Positions  and 
Controlling  Fires 


Common  Preparation  Considerations 


Prepare  Machine  Gun  Positions 


Prepare  Antitank  Weapon  Positions 


This  frame  depicts  the  results  of  the 
assessment.  Note  the  dashed  red  box 
provides  the  recommended  post  assessment 
training  topics.  This  recommendation  to  the 
learner  is  based  on  answer  selections  and  missed 
questions. 


This  frame  depicts  all  available  training 
topics.  Note  the  dashed  red  box  in  the 
previous  frame  recommended  training 
topics.  The  learner  would  be  free  to 
explore  any  available  topic. 


This  practice  (with  assessment)  makes  the  best  use  of  the  students'  time,  focusing  their  learning  on  the  subject  areas 
or  subtasks  in  which  their  demonstrated  performance  falls  below  standards  or  indicates  shortcomings.  It  also  rewards 
their  good  performance  and  shows  where  they  meet  standards. 

Standards,  weighted  factors,  and  scoring  thresholds  can  be  adjusted  in  the  programming  to  assure  that  learning  goals 
are  being  attained. 
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Provide  Assessment  Feedback 

(continued) 

Provide  feedback  for  individual  questions  on  selected  tests  and 
for  checks  on  learning.  Feedback  should  be  provided  in  a 
manner  that  continues  and  reinforces  learning.  Reinforce 
correct  responses.  For  incorrect  responses,  identify  the  error, 
indicate  the  correct  response,  and  provide  supporting  key 
information  to  improve  understanding. 
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Test  Design  is  a  Key  Element  of  Successful  Assessment 


The  best  methods  of  assessment  generally  involve  live  performance 
or  demonstration  of  the  knowledge  or  skill.  However,  time  and 
other  resource  constraints  may  drive  you  to  paper  based  or  media 
embedded  assessments.  There  are  some  simple  rules  that  can  be 
used  to  provide  a  rigorous  assessment  and  maintain  realism. 

•  As  with  all  practice,  problem  based  scenarios  provide  the  best 
context  for  assessments. 

•  If  the  learner  cannot  actually  or  virtually  perform  the  task,  put  the 
learner  in  the  role  of  evaluating  the  performance,  results,  or 
products  of  others.  Have  the  learner  detect  errors  and  problems, 
identify  remedial  actions,  or  determine  correct  solutions. 

•  Avoid  memorization  and  regurgitation.  Learners  often  "memory 
dump"  memorizations  that  are  not  useful  or  necessary  knowledge 
soon  after  training. 

•  Use  question  types  in  a  manner  that  permits  a  demonstration  of 
learner  understanding: 

o  True/False  questions  should  be  avoided, 
o  Select  multiple  choice  responses  that  are  potentially  confused 
with  correct  responses  -  eliminate  throw  away  and  illogical 
responses. 

o  Use  multiple  or  all  correct  responses  for  multiple  choice 
questions. 

o  Use  "not  used"  responses  or  multiple  correct  responses  for 
matching  questions, 
o  Avoid  patterned  responses. 

Standards,  weighted  factors,  and  scoring  thresholds  can  be 
adjusted  in  the  programming  to  assure  that  learning  goals  are 
attained. 
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Assessment  of  Learning:  Test  Design  and 
Question  Examples 

(continued) 


These  tests  and  their  related  questions  (inactivated)  provide  examples  of 
scenarios  and  problems  related  to  skills  that  Soldiers  would  employ  to  find 
solutions  to  real-world  problems.  They  assess  the  practical  knowledge  and 
skills  that  an  Infantry  Rifle  Squad  Leader  or  Fire  Team  Leader  would 
require  to  be  successful. 

They  also  provide  examples  of  balanced/ parallel  pre-  and  post-training 
tests.  Varied  situations  and  tasks  examine  proficiency  in  knowledge 
and  skills  without  duplications.12 


Squad  Urban  Defense 


Engagement  of  Targets  with 
I  ndirect  Fire 


B 


Urban  Def  Test  1 
(KEY) 


Urban  Def  Test  2 
(KEY) 


CFF  Test  1  (KEY) 


CFF  Test  2  (KEY) 
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Conclusion 


Recent  research  indicates  that  there  are  practical  principles  that 
can  enable  the  application  of  CBT  for  individualized  training. 
Other  principles  exist  for  applying  instructional  design  principles 
to  CBT  and  for  CBT  instructional  strategies.  This  Guide 
synthesizes  and  adapts  these  principles  to  Army  instructional 
needs.  The  mission  of  the  Guide  is  to  bridge  the  gap  between 
theory  and  practical  application.  It  provides  guidance  and  tools 
for  Army  trainers  and  training  developers  to  support  the  design 
of  effective  CBT  for  military  instructional  purposes. 

The  Guide  contains  three  primary  sections: 

•  ADDIE  Model  -  The  accepted  framework  and  guide  for 
the  design  and  development  of  effective  CBT. 

•  Design  and  Development  Principles  -  A  series  of  easily- 
referenced  fundamentals  with  examples  that  can  lead  to 
the  design  and  implementation  of  more  effective  learner¬ 
centric  training. 

•  Assessments  -  Fundamentals  that  support  effective  test 
design  and  feedback  with  examples  of  useful  tests, 
questions,  and  feedback  techniques. 

The  section  on  Design  and  Development  Principles  supports  the 
development  of  effective  CBT.  The  principles  focus  on 
generalizable  strategies  and  are  supported  by  relevant  research 
literature,  expert  knowledge,  and  practical  experience.  Those 
Principles  are  summarized  below: 

•  Define  the  learning  task  -  what  the  learner  should  know 
or  be  able  to  do  after  instruction. 

•  Link  new  knowledge  and  skills  to  real  world  tasks  and 
missions  in  context  (problem  based) 

•  Encourage  practice  of  new  knowledge  and  skills  in 
context: 

■  Provide  varied  situations,  progressive  levels  of 
difficulty,  and  lessen  instructional  scaffolding  as 
familiarity  with  task  and  steps  increases. 

■  Provide  performance  feedback  and  tailored 
remediation. 
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Conclusion 

(continued) 


•  Provide  content  in  logical  chunks 

O  Promote/ manage  essential  processing 

•  Segmenting  -  Present  materials  and  animation  in  Learner- 
paced  segments 

•  Pre-training  -  Provide  pre-training  for  essential  basics  and 
prerequisites 

•  Modality  -  Present  narrations  vs.  printed  text 
O  Reduce  extraneous  processing 

•  Coherence  -  Avoid  presenting  extraneous  materials 

•  Signaling  -  Highlight  essential  materials 

•  Redundancy  -  Do  not  add  on-screen  text  to  narrated 
animation 

•  Spatial  Contiguity  -  Place  on-screen  text  next  to 
corresponding  graphics 

•  Temporal  Contiguity  -  Present  corresponding  narration  and 
animation  at  the  same  time 

•  Provide  a  meaningful  learning  experience  -  promote  generative 
processing 

□  Multimedia  -  Present  words  and  pictures  rather  than  words 
alone 

□  Personalization  -  Present  words  in  a  conversational  style 
rather  than  formal  style 

□  Voice  -  Put  narrations  and  dialog  in  human  voice  rather  than 
machine  voice 

□  Embodiment  -  Have  on-screen  agent  use  human- like  gestures 
and  movements 

□  I  mage  -  Do  not  necessarily  put  static  image  of  agent  on  the 
screen 
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APPENDIX  B 

User’s  Guide 
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User’s  Guide  for  Developing  Computer-Based  Training  (CBT): 

Considerations  and  Tips 

Prepared  by 
Victor  J.  Ingurgio,  ARI 
victor  .j  .ingurgio.  civ  @  mail.mil 


The  Army  Research  Institute  (ARI)  constructed  a  summary  of  key  principles  to  guide  the 
development  of  effective  CBT  content.  The  “CBT  Development  Guide”  (version  1.5)  is 
presented  as  a  Microsoft  PowerPoint  presentation.  These  principles  are  based  on  pedagogical 
theory,  content,  and  learner  issues.  Adhering  to  these  principles  will  yield  content  that  will  help 
students  learn. 

To  address  instructor  concerns  about  formats,  styles,  and  technical  development,  this  document 
supplements  the  CBT  Development  Guide.  This  User’s  Guide  includes  some  pre-development, 
developmental,  some  general  design  tips,  and  other  considerations  for  the  design  and 
development  process  of  building  CBT  modules.  Much  of  this  User’s  Guide  material  comes  from 
human  factor  design  standards  (FAA1  and  NASA2)  and  from  subject  matter  expert  (SME) 
recommended  internet  sources3,4’5,6.  These  materials  provide  a  novice  developer  with  some 
additional  resources  and  materials  to  assist  with  the  development  efforts.  This  is  not  an 
exhaustive  list. 

Please  keep  in  mind  that  while  anyone  can  follow  and  comply  with  this  User’s  Guide  to  help 
design  and  develop  CBT,  the  learning  value  could  be  minimal  or  nonexistent  if  the  learning 
principles  outlined  in  the  CBT  Development  Guide  (v  1.5)  are  not  used  to  guide  the  presentation 
of  the  content. 

ARI’s  experience  with  CBT  development  and  feedback  from  SMEs  indicates  that  technical 
development  and  style  design  are  very  project  specific.  That  is,  CBT  design  and  development  is 
dependent  on  the  goals  and  materials,  knowledge,  and  skills  to  be  conveyed  to  the  learner. 
Further,  SMEs  agree  that  the  design  of  a  universal  checklist  or  a  step-by-step  process  for  a 
novice  to  design  and  develop  meaningful  CBT  is  difficult  if  not  impossible  to  compile.  Taking 
this  as  a  challenge,  the  following  User  Guide  is  only  an  attempt  to  consolidate  a  few  good 
sources  for  the  sole  purpose  of  providing  the  novice  developer  some  other  mechanism  to 
consider  when  developing  CBT. 
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Pre-Development  Considerations 


Note:  The  below  considerations  parallel  the  guidance  provided  in  the  Systems  Approach  to 
Training  analysis  found  in  the  Army  TRADOC  Pamphlet  350-70-67  (Department  of  the  Army 
Headquarters,  7  September  2004). 

•  Define  the  target  population 

The  target  population  is  the  group  of  learners  for  whom  the  training  is  intended.  It  is  critical  to 
define  this  group  in  order  to  design  the  training  appropriately.  For  example,  training  for 
officers  would  be  very  different  from  training  for  new  Soldiers,  even  though  they  may  do  some 
of  the  same  tasks.  To  define  the  target  population,  ask  questions  such  as: 

•  What  are  the  job  titles  of  the  intended  participants  in  the  training? 

•  How  were  they  originally  trained  for  their  jobs? 

•  What  are  their  educational  and  professional  backgrounds? 

•  Are  they  still  in  school  or  already  on  the  job? 

•  How  are  they  accustomed  to  learning? 

•  What  languages  do  they  speak  and  read? 

•  What  types  of  facilities  do  they  work  in,  and  how  are  these  facilities  equipped? 

•  By  whom  are  they  supervised? 

•  Is  it  possible  for  them  to  attend  a  training  course  away  from  their  jobs? 

•  List  the  tasks  to  be  performed  by  the  target  population  (For  the  Army,  official  task 
analyses  are  performed  by  the  Critical  Task  and  Site  Selection  Board;  see  Army  TRADOC 
Pamphlet  350-70-6  for  more  details7). 

To  list  the  tasks  to  be  performed  by  the  target  population,  one  must  know  what  “good 
performance”  is;  in  other  words,  what  a  good  performer  would  do  on  the  job.  To  find  out,  the 
training  developer  must  have  access  to: 

•  technical  experts  who  can  accurately  describe  the  job, 

•  good  performers  who  can  be  observed  doing  the  job,  and/or 

•  documents  and  manuals  that  accurately  describe  the  job. 

Through  discussion  with  experts,  observations,  and  review  of  documents,  the  training  developer 
can  develop  a  task  list. 

•  List  the  skills  and  knowledge  needed  to  perform  the  tasks 

For  each  task  involved  in  a  job,  the  training  developer  should  list  the  skills  and  knowledge 
required  to  perform  the  task.  Skills  are  generally  actions  such  as  measuring,  mixing,  and 
recording,  calculating,  communicating,  or  making  decisions.  Required  knowledge  is  the 
information  needed  to  do  a  task  correctly. 
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Select  the  skills  and  knowledge  to  be  taught  (i.e.,  the  training  objectives) 


Experts  use  a  list  of  criteria  to  decide  which  skills  and  knowledge  to  include  in  the  training. 

These  will  make  up  the  training  objectives  for  the  course.  The  first  list  below  shows  factors 
that  would  lead  to  inclusion  in  the  course;  the  second  list  shows  factors  that  would  suggest 
that  the  skill  or  knowledge  could  be  excluded  (not  taught)  in  the  course.  Some  of  the  factors 
may  vary  in  relevance  in  different  situations. 

Possible  criteria  for  inclusion 

•  Many  members  of  the  target  population  lack  the  skill  or  knowledge. 

•  Training  (including  practice  and  feedback)  is  required  to  learn  the  skill  or  knowledge 
because  it  is  new  or  difficult. 

•  The  task  for  which  the  skill  or  knowledge  is  needed  is  important  to  the  outcome. 

•  The  skill  or  knowledge  is  needed  frequently. 

•  It  is  practical  to  teach  the  skill  or  knowledge  in  the  given  training  setting. 

Possible  criteria  for  exclusion 

•  The  task,  skill,  or  knowledge  cannot  be  described  specifically  and  thoroughly  enough  to  be  a 
meaningful  part  of  training.  (This  may  be  because  of  differences  of  opinion  among  technical 
experts,  lack  of  authoritative  evidence  on  how  the  task  should  best  be  done,  etc.). 

•  Teaching  the  skill  or  knowledge  is  not  practical  in  the  time  or  with  the  resources 
available. 

•  Most  members  of  the  target  population  already  have  the  skill  or  knowledge. 

•  The  task,  skill,  or  knowledge  is  straightforward  and  could  be  done  correctly  after  reading 
guidelines,  such  as  a  checklist  or  manual.  Practice  and  feedback  are  not  required. 

•  The  task  is  done  or  the  skill/knowledge  is  used  infrequently  (e.g.,  it  deals  with  a  situation 
that  is  extremely  rare). 

•  The  task  is  done  differently  in  different  areas  or  other  facilities  (must  be  tailored  to  a 
unique  setting). 

•  The  task  for  which  the  skill  or  knowledge  is  needed  is  of  low  importance. 

•  There  are  substantial  obstacles  to  doing  the  task  (e.g.,  lack  of  equipment,  tools,  or  time) 
that  would  prevent  the  learner  from  doing  the  task  even  if  he/she  knew  how.  These 
obstacles  would  have  to  be  overcome  before  the  training  could  be  useful. 

•  Another  training  course  is  available  to  teach  the  task/skill/knowledge. 

Similar  criteria  can  be  used  to  decide  which  of  the  included  tasks,  skills,  and  knowledge  will 
receive  more  emphasis  and  practice  in  the  training  course. 
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Design  Considerations 


As  part  of  the  design  process,  the  training  developers  organize  the  selected  skills  and 
knowledge  to  be  taught  into  logical  teaching  units  called  modules.  The  design  for  each  module 
includes  its  training  objectives  and  a  brief  outline  of  the  information,  examples,  and  exercises 
that  will  provide  opportunities  for  practice  using  the  skills  and  knowledge  (see  CBT 
Development  Guide). 

The  design  of  each  module  progresses  from  the  brief  design  outline,  to  an  expanded  outline,  to 
the  complete  module.  Expanded  outlines  of  the  modules  specify  more  completely  the 
information  and  the  types  of  examples  and  exercises  to  be  provided.  Examples  might  be  given 
through  pictures,  live  demonstrations,  or  video.  Exercises  might  include  written  exercises, 
group  discussions,  role  playing,  or  practice.  To  develop  realistic  examples  and  exercises,  the 
training  developers  rely  on  interviews  with  technical  experts  who  are  familiar  with  the  target 
population,  job  setting,  tasks,  and  conditions. 

•  Instructional  modules 

An  instructional  module  contains  the  teaching  materials,  such  as  text,  pictures,  video,  etc.  The 
design  of  instructional  materials  must  involve  the  design  of  each  individual  item  (text,  graphics, 
audio,  video,  etc.)  as  well  as  ensuring  that  for  each  learning  objective,  the  collection  of 
instructional  items  form  a  coherent  collection  that  will  facilitate  the  student’s  learning  (see  CBT 
Development  Guide  for  information  on  chunking,  organizing,  and  sequencing  material). 

Another  consideration  in  the  design  of  instructional  modules  is  the  detailed  design  of  the  various 
instructional  materials  for  each  learning  objective.  It  is  assumed  that  content  design  has 
preceded  CBT  design  and  has  proceeded  to  the  point  that  all  learning  objectives  have  been 
identified. 

•  Formative  tests 

A  formative  test  is  used  to  provide  the  student  with  a  measure  of  an  understanding  of  the  subject 
being  presented.  In  order  to  provide  the  student  with  frequent  measures  of  understanding, 
formative  tests  should  occur  often  and  should  cover  small  amounts  of  material. 

Format  of  the  presentation  to  the  student,  type  of  feedback,  and  timing  of  feedback  are  important 
design  considerations.  Some  combinations  of  the  following  options  are  possible,  but  not  all 
work  well  together  and  some  induce  significant  development  costs  (see  Development 
Consideration  below). 

•  Presentation  options 

•  One  question  per  page 

•  All  questions  on  one  page  with  questions  grouped  according  to  type  (T/F,  fill-in-the- 
blank,  multiple  choice,  etc.) 


B-5 


•  All  questions  on  one  page  with  questions  not  grouped  according  to  type  (T/F,  fill-in- 
the-blank,  multiple  choice,  etc.) 

•  Separate  pages  for  each  type  of  question 

•  Feedback  options 

•  Correct  or  incorrect 

•  Provide  reward 

•  Provide  reward  with  additional  challenge 

•  Provide  correct  answer 

•  Suggested  remedial  study 

•  Provide  remedial  instruction 

•  Provide  score 

•  Reasonable  combinations  of  the  above 

•  Timing  options 

•  After  each  question 

•  After  some  predefined  group  of  questions 

•  After  the  test  has  been  completed 

The  design  of  formative  tests  also  requires  a  decision  about  how  an  individual  test  is  constructed. 
For  any  given  formative  test,  there  are  at  least  four  choices  for  test  construction: 

•  The  same  pre-constructed  test  is  presented  each  time 

•  The  test  is  selected  (randomly  or  by  rotation)  from  a  collection  of  pre-constructed 
tests 

•  The  test  is  constructed  by  randomly  (this  may  involve  weighting)  choosing  questions 
from  a  collection  of  questions. 

•  The  test  is  constructed  by  assembling  computer-generated  questions 

•  Evaluative  tests 

Evaluative  tests  are  primarily  for  reporting  purposes.  They  are  designed  to  evaluate  and  record 
the  student’s  mastery  of  a  subject.  Evaluative  tests  occur  less  frequently  than  formative  tests  and 
evaluate  the  student’s  mastery  of  large  blocks  of  the  subject  matter. 

Development  Considerations 

•  Content  development 

Developing  course  content  consists  of  constructing  and/or  assembling  teaching  materials  for 
each  learning  objective  as  prescribed  by  the  design  of  instructional  materials.  Fundamental 
qualifications  of  persons  performing  content  development  include  SMEs  with  a  solid  foundation 
in  the  principles  of  teaching  and  learning  (see  CBT  Development  Guide  vl.5).  Beyond  these 
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two  fundamental  qualifications,  the  development  team  must  possess  expertise  in  the  various 
media  (writing,  graphics,  audio,  video,  animation,  etc.)  being  used,  as  well  as  the  computer  tools 
used  to  create  the  media. 

The  cost  information  below  may  not  be  relevant  to  all  designers.  It  is  added  here  for  general 
consideration  purposes. 

•  Development  cost 

The  cost  of  development  is  most  frequently  quoted  in  terms  of  development  time  required  to 
develop  1  hour  of  CBT  time,  which  itself  is  impossible  to  quantify.  Estimates  range  from  as  low 
as  100  hours  of  development  time  per  hour  of  CBT  time  to  a  high  of  1,000  hours  of  development 
time  per  hour  of  CBT  time.  In  most  of  these  estimates,  the  cost  of  actually  producing  the  media 
was  not  included  in  the  estimate. 

How  does  one  calculate  the  cost  of  CBT  development?  The  flippant  answer  is  to  try  to  calculate 
the  cost  after  the  project  is  complete.  For  a  more  acceptable  answer,  one  should  adhere  to  the 
following  general  guidelines  during  the  design  process  and  develop  a  cost  estimate  at  the  end  of 
the  design  process. 

•  Cost  control  guidelines 

These  guidelines  are  generalities  and  it  should  be  expected  that  each  individual  situation  will 
impose  its  own  unique  modifications  on  these  guidelines. 

•  Design  the  CBT  system  to  meet  its  intended  purpose. 

•  Adding  an  option  will  increase  cost. 

•  Selecting  options  that  the  developer  has  previously  implemented  will  reduce  the 
cost  increase. 

•  Changes  late  in  the  development  process  are  extremely  expensive. 

•  Experienced  developers  may  use  a  higher  base  rate,  but  total  cost  is  generally  less 
than  the  total  cost  resulting  from  an  inexperienced  developer  with  a  lower  base 
rate. 

•  Every  dollar  spent  on  design  saves  three  to  five  dollars  in  development  costs. 

•  Computer  program  development  is  a  creative  process  and  not  all  time  (costs)  can 
be  accurately  predicted. 

•  Permit  at  least  10%  of  the  total  development  cost  to  be  devoted  to  testing  the 
product 
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Maintenance  costs  of  a  poorly  designed  or  poorly  implemented  system  can  exceed 
the  cost  of  initial  development 


For  larger  systems,  it  is  generally  cost  effective  to  initially  develop  a  comprehensive 
management  system  (content  management,  student  management)  so  that  no  programming  costs 
are  involved  in  the  development  of  the  actual  instructional  modules.  In  a  system  consisting  of 
100  modules  (hours  of  CBT  time)  or  more,  this  approach  should  reduce  total  cost  per  module  to 
the  range  of  50  hours  per  hour  of  CBT  time. 

Another  consideration  is  to  construct  a  development  schedule.  To  construct  such  a  schedule,  the 
following  considerations  are  important: 

•  Required  resources 

•  Degree  to  which  the  instructor  will  participate 

•  Resources  available  from  the  instructor 

•  Resources  available  from  the  developer 

•  Events  that  are  anticipated  to  interfere  with  development 

•  Marketing  considerations 

Although  the  needs  and  desires  of  the  instructor  are  determining  factors,  the  developer  is  able  to 
provide  realistic  estimates  of  required  time,  and  for  that  reason  input  may  dominate  during 
construction  of  the  schedule.  It  should  be  clear  from  the  descriptions  above  that  both  the 
instructor  and  the  developer  must  be  involved  in  the  design  of  each  component. 

Checklist  Options 

The  following  material  could  be  used  by  novice  CBT  developers  for  designing  an  in-house  “style 
guide”,  and  may  serve  as  a  checklist  to  ensure  the  developed  material  complies  with  the  CBT 
Development  Guide. 

•  Course  organization 

Each  course  should  include  the  following  elements.  Check  off  each  item  on  the  list  as  you 
address  it: 


•  course  title  screen 

•  welcoming  screen 

•  course  menu  that  uses  numbers  or  letters 

•  directions  on  how  to  use  the  CBT  system,  how  to  sign  off,  and  how  to  return 

•  course  objectives 

•  course  prerequisites 

•  the  duration  of  the  course 

•  references  to  supporting  materials  such  as  workbooks  or  user’s  guides 

•  an  “escape”  option  to  help  the  user  exit  from  any  screen  or  return  to  a  screen 
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•  completion  status  of  each  topic 

•  backward  paging  to  give  more  control  over  the  learning  process 

•  an  introductory  screen  containing  the  topic  name,  a  brief  description  of  the  content, 
and  a  sentence  or  two  of  transition  or  orientation  that  links  it  to  what  has  preceded  it 
and  what  will  follow  it 

•  a  list  of  objectives 

•  summary  screens 

In  addition,  course  developers  should  keep  the  following  points  in  mind.  Again,  check  off  each 
item  on  the  list  as  you  make  sure  you  have  addressed  it: 

•  No  unit  should  be  longer  than  20  or  30  minutes.  Consider  using  menus  and  submenus 
to  divide  long  topics. 

•  Avoid  timed  displays,  even  those  that  are  timed  to  display  for  more  than  a  few 
seconds.  (The  only  exception  is  a  timed  display  that  builds  a  final  image  that  remains 
on  the  screen  until  the  student  presses  a  key  to  move  on.) 

Timed  displays  are  a  problem  for  several  reasons.  For  instance,  students  who  turn  away  from  the 
computer  for  even  a  few  seconds  may  miss  critical  information;  timed  displays  take  control  away 
from  the  learner.  Also,  they  may  distract  slow  readers  by  throwing  new  information  onto  the 
screen  in  different  places.  Further,  they  can  frustrate  faster  readers  who  prefer  to  move  at  their 
own,  speedier,  rates. 

•  Screen  composition 

Each  course  should  contain  the  following  screen  elements: 

•  unit  or  sub-unit  identification — on  the  first  screen  of  each  unit  or  sub-unit 

•  a  screen  header  and  footer  area,  containing  unit  and  screen  numbers 

•  consistent  fonts,  type  sizes,  punctuation,  and  placement 

•  blank  space  at  the  top,  bottom,  and  side  margins  to  avoid  clutter 

•  one  main  idea  per  screen 

•  bulleted  lists  of  ideas 

•  paragraphs  of  no  more  than  six  to  eight  lines,  if  possible 

•  single  spacing  within  paragraphs,  and  double  spacing  between  the  paragraphs 

•  no  indents  for  paragraphs,  except  with  bullets 

•  boxes  containing  instructional  information,  of  the  same  size  and  in  a  consistent 
location  on  the  screen 

•  typing  directions  in  the  boxes 

•  all-capital  letters  used  only  for  warnings  or  hints. 
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Graphics  and  Fonts 


•  Have  you  only  used  a  maximum  of  four  fonts  throughout  the  entire  course? 

•  Is  the  body  text  in  the  same  font,  and  decorative  text  only  used  in  the  headers? 

•  Have  you  included  the  right  line  spacing  and  paragraph  length? 

•  Have  you  used  the  correct  font  sizes  and  colors  (are  they  all  visible)? 

•  Is  the  body  text  left  justified? 

•  Have  you  included  a  minimal  amount  of  bold/italicized  words? 

•  Are  the  images  relevant,  compressed  and  re-sized?  Do  you  legally  own  all  of 
them  (or  have  copyright  authorization)? 

•  Are  all  of  the  images  the  right  file  type  and  consistent  (in  terms  of  quality  and 
size)? 

•  Are  screen  captures  free  of  personal  data  and  current? 

•  Multimedia 

•  Have  you  included  relevant  and  legally  owned  (or  copyright  authorization) 
images  and  video  elements  in  the  course? 

•  Are  all  files  compressed  for  easy  download? 

•  Are  all  of  the  multimedia  elements  consistent  (in  terms  of  size  and  quality)? 

•  Is  the  audio  synced  to  the  video  presentations? 

•  Is  the  narration  clear  and  easy  to  understand? 

•  Have  you  ensured  that  the  audio  or  video  can  be  controlled  by  the  user  (e.g., 
paused)? 

•  Accessibility 

•  Have  you  used  all  of  the  ALT  tags  available? 

•  Are  captions  included  for  all  audio  or  video  elements  that  deal  with  core 
content? 

•  Are  all  of  your  fields  and  clickable  areas  easy  to  detect  and  use? 

•  Navigation 

•  Is  the  course  easy  to  navigate  for  the  learner? 

•  Can  you  easily  navigate  through  the  entire  course  using  just  the  keyboard  and 
mouse? 

•  Are  all  course  modules  featured  in  their  correct  sequence? 

•  Are  hyperlinks  displayed  clearly  and  do  they  all  work  properly? 
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•  Have  you  included  backward  links  so  that  learners  can  return  to  previous 
screens? 

•  Is  there  an  abundance  of  navigation  graphics?  If  so,  whittle  down  to  only 
what  is  needed. 

•  Is  there  a  table  of  contents  and  a  map  of  the  course? 

•  Technical  Issues 

•  Does  the  course  load-up  within  a  reasonable  time  frame? 

•  Are  shortcut  keys  available  to  the  learner? 

•  Do  you  have  an  FAQ  page  with  hyperlinks  to  other  areas  of  the  course? 

•  Have  all  hardware  and  software  requirements  and  recommendations  been 
listed? 

•  Have  the  dimensions  (display  screen  size)  of  the  course  been  optimized  for 
users? 

•  Is  the  course  viewable  in  all  of  the  major  web  browsers? 

•  Can  selected  pages  (e.g.,  summary  pages,  or  score  sheets)  be  printed  with 
ease? 

•  Are  assessments  being  scored  properly? 

•  Overall  Design 

•  Do  you  have  a  clear  syllabus  in  place  at  the  beginning  of  the  course? 

•  Is  the  course  aesthetically  uniform  and  have  you  included  branding  elements 
throughout? 

•  Is  the  color  usage  consistent  and  appropriate? 

•  Are  all  of  the  elements  visible  with  the  chosen  color  scheme? 

•  Does  at  least  half  the  screen  consist  of  white  space,  to  keep  the  look  clean  and 
organized? 

•  Have  you  verified  that  all  content  fits  the  screen  (no  horizontal/vertical 
scrolling)? 

•  Have  you  avoided  backgrounds/patterns  that  may  be  distracting  for  the 
learner? 

•  Can  instructors  easily  add  or  modify  the  content  within  the  course? 
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Footnotes 


1  Human  Factors  Design  Standard;  DOT/FAA/CT-03/05  HF-STD-001  (2003). 

2Human  Integration  Design  Handbook;  NASA/SP-201 0-3407/REV  1  (2014). 
3https://www.questia.com/magazine/lG  1  - 1 56 1 1 5 17/a-common-sense-checklist-for-cbt 
4http://elearningindustry.com/the-ultimate-elearning-course-design-checklist 
5http://whqlibdoc.who.int/hq/2005/WHO_HTM_TB_2005. 354_partl_eng.pdf 
6http://www.delweg.com/dpwessay/cbtdvlop.htm 
7http://tradoc. army.mil/tpubs/pams/p350-70-6.pdf 
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APPENDIX  C 

Design  Assessment  Worksheet  for 
Interactive  Multimedia  Instruction  (IMI) 
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Design  Assessment  Worksheet 
for 

Interactive  Multimedia  Instruction  (IMI) 


The  purpose  of  this  worksheet  is  to  aid  leaders,  trainers  and  writers,  training  developers,  and 
media  technicians  (artists  and  programmers)  in  assessing  the  design  of  interactive  multimedia 
instruction  (IMI).  It  is  focused  on  assessing  how  effectively  sound  design  principles  and 
techniques  have  been  integrated  into  the  IMI  to  enhance  learning. 

Worksheet  users  should  possess  an  understanding  and  working  knowledge  of  the  learning 
principles  and  techniques.  The  worksheet  is  designed  to  be  used  in  conjunction  with  the 
Computer-Based  Training  Principles  Guide,  a  research  product  of  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences.  For  each  “No”  response,  written  remarks 
should  be  provided  to  cite  specific  examples  or  to  provide  recommendations  for  improvement. 

1.  Does  the  IMI  state  the  Terminal  Learning  Objective  (task,  conditions,  and  standards)  for  the 
block  of  training  (course,  lesson,  module,  or  topic)  and/or  the  task(s)  to  be  learned? 


Remarks: 


2.  Does  the  IMI  state  the  Enabling  Learning  Objective  (task,  conditions,  and  standard)  for  the 
block  of  training  (course,  lesson,  module,  or  topic)  and/or  the  tasks(s)  to  be  learned? 


Remarks: 


3.  Does  the  IMI  relate  new  tasks  (knowledge  or  skills)  to  previously  learned  tasks  OR  give  a  job- 
related  reason  to  learn  the  task? 


Remarks: 


4.  Does  the  IMI  present  the  instruction  in  the  context  of  real  world/job-related  problems  or  tasks? 


Remarks: 
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5.  Does  the  IMI  provide  the  opportunity  to  practice/display  the  new  knowledge  or  skill  gained? 
(Dedicated  practice,  user  self-checks,  checks  on  learning)? 


Remarks: 


5.  A.  Does  the  IMI  provide  feedback  for  checks  on  learning,  self-assessments,  and 
practice? 


Remarks: 


5.  B.  Does  the  IMI  feedback  include  an  explanation  or  information  addressing  the 
learner’s  errors  or  mistakes? 


Remarks: 


5.  C.  If  the  user  failed  to  meet  the  standards  of  the  assessments,  practice,  or  checks  on 
learning,  does  the  IMI  allow  the  student  to  review  the  topic  or  receive  retraining? 


Remarks: 


5.  D.  Do  the  checks  on  learning,  assessments,  or  practice  present  varied  circumstances, 
situations,  and  problems  in  which  the  user  can  apply  the  new  knowledge  and  skills? 


Remarks: 


5.  E.  Do  the  checks  on  learning,  assessments,  or  practice  provide  increased  complexity  or 
difficulty  to  provide  or  encourage  the  learner’s  confidence  in  the  use  of  new  knowledge 
and  skills?  (Crawl,  Walk,  Run) 


Remarks: 
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5.  F.  Does  the  IMI  provide  a  post- test  with  feedback  to  determine  student’s  level  of 
proficiency? 


Remarks: 


5.  G.  Does  the  post  training  assessment  cover  key  teaching  points,  ELOs,  and/or  the 
stated  performance  standards? 


Remarks: 


6.  If  the  new  knowledge  and  skills  relate  to  subjects  that  the  user  has  prior  knowledge  or  related 
experience  with,  does  the  IMI  provide  a  pretest  or  pre-training  assessment  to  determine  the  level 
of  user  knowledge  prior  to  training? 


Remarks: 


7.  Does  the  IMI  provide  or  review  “up-front”  or  prerequisite  knowledge  prior  to  getting  into  the 
details  of  task,  knowledge,  or  skills?  (Pretraining) 


Remarks: 


7.  A.  If  there  are  elements  of  knowledge  or  skills  such  as  terms,  part  nomenclatures, 
tools,  skills  (or  skill  refreshers),  or  foundations  that  the  learner  requires,  does  the  IMI 
provide  them  or  references  to  them? 


Remarks: 


8.  Does  the  IMI  present  the  topics,  modules,  or  lessons  in  small  manageable  chunks,  segments, 
or  pieces?  (Segmented) 


Remarks: 
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9.  Does  the  IMI  provide  the  learner  with  the  means  (navigation,  replay  button,  etc.)  to  control 
the  pace  of  presentation? 


Remarks: 


10.  Does  the  IMI  present  or  convey  the  instruction  in  a  conversational  manner  and  tone  -  as  a 
live  person  would  if  he  was  present  and  talking  with  or  presenting  information  to  the  student? 
(Modality) 


Remarks: 


10.  A.  Is  the  narration  understandable?  (Voice) 


Remarks: 


10.  B.  Is  the  narration  done  in  a  conversational  or  “polite”  style?  (Could  the  user 
develop  a  social  partnership  with  the  narrator,  or  does  it  sound  as  though  the  narrator  is 
reading  instructions  from  a  technical  publication?)  (Personalization) 


Remarks: 


10.  C.  Does  the  narration  avoid  a  verbatim  reading  of  text  displayed  on  the  screen? 
(Redundancy) 


Remarks: 


10.  D.  Is  the  narration  synchronized  with  video  clips,  animation,  and  sequenced  graphics? 
Does  the  narration  match  with  and  support  what  the  user  sees  on  the  screen?  (Temporal 
contiguity) 


Remarks: 


C-5 


11.  Does  the  IMI  stay  on  the  topic  and  focus  on  learning  objectives;  is  there  a  lot  of  “filler”  or 
extra  (nice  to  know)  material?  (Coherence) 


Remarks: 


12.  Does  the  IMI  highlight  important  information  in  a  topic  and  clearly  delineate  steps  in 
processes?  (Signaling) 


Remarks: 


12.  A.  Does  the  IMI  have  some  standard  means  to  let  the  user  know  when  something 
critical,  essential,  or  important  is  being  conveyed? 


Remarks: 


13.  Does  the  IMI  label  appropriate  components  in  graphic  displays  in  a  nonintrusive  manner,  if 
or  when  required?  (Spatial  contiguity) 


Remarks: 


14.  Does  the  IMI  provide  a  meaningful  learning  experience;  does  it  include  all  the  required 
learning  material? 


Remarks: 


15.  Does  the  IMI  present  learning  materials  with  graphics  (images,  animations,  and  video  clips) 
and  narrations,  and  present  only  limited  or  supplemental  on-screen  text?  (Is  multimedia  used  to 
convey  learning  or  is  it  a  book  in  a  box?)  (Multimedia) 


Remarks: 


C-6 


16.  Does  the  IMI  use  a  static  “talking  head”,  on  screen  narrator,  or  agent  for  emphasis? 
(Embodiment) 


Remarks: 


C-7 


